SSL issues with external resources

[Andy Bohne]

I am trying to configure Rundeck to use a URL to retrieve a list of nodes from Foreman, which is running on my puppet master.

I've got the SSL certificate from Foreman imported into /etc/rundeck/ssl/keystore and /etc/rundeck/ssl/truststore (both are actually identical).  I've got the resource URL configured as follows:

https://rundeck:run...@puppet01.company.com/hosts?rundeck=true&format=yaml

However, when I go to the nodes tab, I don't see the expected nodes.

In rundeck.log I see the following:

2014-02-24 10:52:49,355 [qtp1573703228-58] ERROR com.dtolabs.rundeck.core.resources.ExceptionCatchingResourceModelSource - [ResourceModelSource: 2.url (URL Source), project: admin]

com.dtolabs.rundeck.core.resources.ResourceModelSourceException: Error requesting URL Resource Model Source: https://rundeck:run...@puppet01.company.com/hosts?rundeck=true&format=yaml: com.dtolabs.rundeck.core.common.FileUpdaterException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

SSL is configured as specified in http://rundeck.org/docs/administration/configuring-ssl.html

The certificate in the keystore seems to be correct.

[root@prodrundeck01 ~]# keytool -list -keystore /etc/rundeck/ssl/keystore -alias puppet01.company.com -v

Enter keystore password:

Alias name: puppet01.company.com

Creation date: Feb 21, 2014

Entry type: trustedCertEntry

Owner: CN=puppet01.company.com

Issuer: CN=Puppet CA: puppet01.company.com

Serial number: 2

Valid from: Sun Mar 31 16:51:55 EDT 2013 until: Sat Mar 31 16:51:55 EDT 2018

Certificate fingerprints:

         MD5:  EE:35:4D:B8:35:F0:54:4E:07:35:D3:3A:B1:E5:44:6B

         SHA1: C8:19:58:F5:2A:8D:45:8F:53:6F:C8:C1:87:9D:4D:F7:B0:AB:14:6F

         Signature algorithm name: SHA256withRSA

         Version: 3

Extensions:

#1: ObjectId: 2.5.29.15 Criticality=true

KeyUsage [

  DigitalSignature

  Key_Encipherment

]

#2: ObjectId: 2.5.29.19 Criticality=true

BasicConstraints:[

  CA:false

  PathLen: undefined

]

#3: ObjectId: 2.5.29.37 Criticality=true

ExtendedKeyUsages [

  serverAuth

  clientAuth

]

#4: ObjectId: 2.16.840.1.113730.1.13 Criticality=false

#5: ObjectId: 2.5.29.14 Criticality=false

SubjectKeyIdentifier [

KeyIdentifier [

0000: E3 9F 35 94 5D 4E 1D 01   C3 F4 AF 57 8B 56 30 84  ..5.]N.....W.V0.

0010: 94 B8 77 34                                        ..w4

]

]

#6: ObjectId: 2.5.29.17 Criticality=false

SubjectAlternativeName [

  DNSName: puppet

  DNSName: puppet.company.com

  DNSName: puppet01.company.com

]

I am running rundeck 2.0.1 on RHEL 6.

Any suggestions on what might be wrong?

[Greg Schueler]

Hi Andy,

There are two different goals when configuring Rundeck for "SSL".  

The first, as described at this page: http://rundeck.org/docs/administration/configuring-ssl.html is for configuring the Rundeck *server* to use expose SSL, and allow the Rundeck CLI tools to communicate to the server over SSL.

The second goal is to configure Rundeck to allow it to communicate to *other servers* using the appropriate SSL certificates.

You want to do the latter.

The difference is that you will need to make sure the /etc/rundeck/profile sets the appropriate JVM options to define the keystore/truststore location for the server.  There is a set of "RDECK_SSL_OPTS" defined in /etc/rundeck/profile, but they are not added to the RDECK_JVM value by default.

After the line "export RDECK_SSL_OPTS=..." you should add:

export RDECK_JVM="$RDECK_JVM $RDECK_SSL_OPTS"

Then you would need to *restart* Rundeck server, so that it executes with those JVM options.  

That configuring-ssl page needs to be updated to address that second goal.

We also conflate the JVM options used by the CLI tools, and the Rundeck server within the /etc/rundeck/profile, unfortunately.  That needs a bit of cleanup which hasn't happened yet.

--
Greg Schueler

--
You received this message because you are subscribed to the Google Groups "rundeck-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rundeck-discu...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Andy Bohne]

Thanks for the input Greg.  I added the line that you provided, but unfortunately that now broke my LDAPS authentication. :(

When I attempt to login, I see the following in rundeck.log:

2014-02-24 13:25:40.967:WARN:oejpj.JAASLoginService:

javax.security.auth.login.LoginException: java.lang.IllegalStateException: Unable to establish root context|?at com.dtolabs.rundeck.jetty.jaas.JettyCachingLdapLoginModule.initialize(JettyCachingLdapLoginModule.java:651)|?at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)|?at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)|?at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)|?at java.lang.reflect.Method.invoke(Method.java:597)|?at javax.security.auth.login.LoginContext.invoke(LoginContext.java:756)|?at javax.security.auth.login.LoginContext.access$000(LoginContext.java:186)|?at javax.security.auth.login.LoginContext$4.run(LoginContext.java:683)|?at java.security.AccessController.doPrivileged(Native Method)|?at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)|?at javax.security.auth.login.LoginContext.login(LoginContext.java:579)|?at org.eclipse.jetty.plus.jaas.JAASLoginService.login(JAASLoginService.java:217)|?at org.eclipse.jetty.security.authentication.FormAuthenticator.validateRequest(FormAuthenticator.java:183)|?at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:456)|?at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)|?at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031)|?at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)|?at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)|?at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965)|?at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)|?at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)|?at org.eclipse.jetty.server.Server.handle(Server.java:349)|?at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:449)|?at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:925)|?at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857)|?at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)|?at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:76)|?at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:191)|?at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609)|?at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45)|?at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)|?at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)|?at java.lang.Thread.run(Thread.java:662)|Caused by: javax.naming.CommunicationException: simple bind failed: dc8.pennmutual.com:636 [Root exception is javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty]|?at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:195)|?at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2720)|?at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:296)|?at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:175)|?at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:193)|?at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:136)|?at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:66)|?at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:667)|?at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:288)|?at javax.naming.InitialContext.init(InitialContext.java:223)|?at javax.naming.InitialContext.<init>(InitialContext.java:197)|?at javax.naming.directory.InitialDirContext.<init>(InitialDirContext.java:82)|?at com.dtolabs.rundeck.jetty.jaas.JettyCachingLdapLoginModule.initialize(JettyCachingLdapLoginModule.java:649)|?... 32 more|Caused by: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty|?at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:190)|?at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1747)|?at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1708)|?at com.sun.net.ssl.internal.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1691)|?at com.sun.net.ssl.internal.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1617)|?at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:105)|?at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)|?at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)|?at com.sun.jndi.ldap.Connection.writeRequest(Connection.java:414)|?at com.sun.jndi.ldap.Connection.writeRequest(Connection.java:387)|?at com.sun.jndi.ldap.LdapClient.ldapBind(LdapClient.java:332)|?at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:190)|?... 44 more|Caused by: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty|?at sun.security.validator.PKIXValidator.<init>(PKIXValidator.java:57)|?at sun.security.validator.Validator.getInstance(Validator.java:161)|?at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.getValidator(X509TrustManagerImpl.java:108)|?at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:204)|?at com.dtolabs.rundeck.jetty.jaas.HostnameVerifyingTrustManager.checkServerTrusted(HostnameVerifyingTrustManager.java:66)|?at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1201)|?at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:135)|?at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:593)|?at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:529)|?at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:943)|?at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1188)|?at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:654)|?at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:100)|?... 50 more|Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty|?at java.security.cert.PKIXParameters.setTrustAnchors(PKIXParameters.java:183)|?at java.security.cert.PKIXParameters.<init>(PKIXParameters.java:103)|?at java.security.cert.PKIXBuilderParameters.<init>(PKIXBuilderParameters.java:87)|?at sun.security.validator.PKIXValidator.<init>(PKIXValidator.java:55)|?... 62 more|

My LDAPS connection was working fine prior to adding export RDECK_JVM="$RDECK_JVM $RDECK_SSL_OPTS" to /etc/rundeck/profile

On Monday, February 24, 2014 1:01:57 PM UTC-5, Greg Schueler wrote:

Hi Andy,

There are two different goals when configuring Rundeck for "SSL".  

The first, as described at this page: http://rundeck.org/docs/administration/configuring-ssl.html is for configuring the Rundeck *server* to use expose SSL, and allow the Rundeck CLI tools to communicate to the server over SSL.

The second goal is to configure Rundeck to allow it to communicate to *other servers* using the appropriate SSL certificates.

You want to do the latter.

The difference is that you will need to make sure the /etc/rundeck/profile sets the appropriate JVM options to define the keystore/truststore location for the server.  There is a set of "RDECK_SSL_OPTS" defined in /etc/rundeck/profile, but they are not added to the RDECK_JVM value by default.

After the line "export RDECK_SSL_OPTS=..." you should add:

export RDECK_JVM="$RDECK_JVM $RDECK_SSL_OPTS"

Then you would need to *restart* Rundeck server, so that it executes with those JVM options.  

That configuring-ssl page needs to be updated to address that second goal.

We also conflate the JVM options used by the CLI tools, and the Rundeck server within the /etc/rundeck/profile, unfortunately.  That needs a bit of cleanup which hasn't happened yet.

--
Greg Schueler

On February 24, 2014 at 8:21:17 AM, Andy Bohne (andy....@gmail.com) wrote:

I am trying to configure Rundeck to use a URL to retrieve a list of nodes from Foreman, which is running on my puppet master.

I've got the SSL certificate from Foreman imported into /etc/rundeck/ssl/keystore and /etc/rundeck/ssl/truststore (both are actually identical).  I've got the resource URL configured as follows:

https://rundeck:rundeck@puppet01.company.com/hosts?rundeck=true&format=yaml

However, when I go to the nodes tab, I don't see the expected nodes.

In rundeck.log I see the following:

2014-02-24 10:52:49,355 [qtp1573703228-58] ERROR com.dtolabs.rundeck.core.resources.ExceptionCatchingResourceModelSource - [ResourceModelSource: 2.url (URL Source), project: admin]

com.dtolabs.rundeck.core.resources.ResourceModelSourceException: Error requesting URL Resource Model Source: https://rundeck:rundeck@puppet01.company.com/hosts?rundeck=true&format=yaml: com.dtolabs.rundeck.core.common.FileUpdaterException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

[Andy Bohne]

Any idea why enabling a truststore would cause my LDAPS connection to break?

If I go back and comment out the export RDECK_JVM="$RDECK_JVM $RDECK_SSL_OPTS" line in /etc/rundeck/profile, LDAPS starts working correctly again.

[Greg Schueler]

Hi Andy,

the "the trustAnchors parameter must be non-empty" usually means that path to the truststore is wrong, or the file is empty...

can you double-check the option paths and the content of the file?

--
Greg Schueler

On February 26, 2014 at 11:51:00 AM, Andy Bohne (andy....@gmail.com) wrote:

Any idea why enabling a truststore would cause my LDAPS connection to break?

If I go back and comment out the export RDECK_JVM="$RDECK_JVM $RDECK_SSL_OPTS" line in /etc/rundeck/profile, LDAPS starts working correctly again.

[Andy Bohne]

That seems to have been the issue.  I modified RDECK_SSL_OPTS as follows:

export RDECK_SSL_OPTS="-Djavax.net.ssl.trustStore=/etc/rundeck/ssl/truststore -Djavax.net.ssl.trustStoreType=jks -Djava.protocol.handler.pkgs=com.sun.net.ssl.internal.www.protocol"

Now I am back in business.  Thanks for the help!

One more question regarding all of this SSL business....

If I wanted to modify this config so that Rundeck is not listening on HTTPS, what would be the appropriate step(s)?

Basically, I'm looking to front Rundeck with apache httpd, and terminate SSL at that point.

[Baron Von J]

I am also interested in being able to use https URLs for the node resources without having to use https for rundeck itself.

[Lars Solberg]

Did any of you ever find out how to allow https resources but not having rundeck be on https? I already have a proxy doing https in front of rundeck..