AttributeQuery failing: CURLSOAPTransport error

[Roy, Nicholas S]

I'm testing our SAML2 AttributeQuery endpoint using a test configuration with signed metadata, with valid IdP and SP SAML signing keys included, with the correct keys for both the SP and IdP in the metadata, and correct endpoints listed. In order to test the AttributeQuery profile of this IdP, I've set the relying party config for the test SP to not release attributes via the front channel (SSO.) I get the following messages in the SP's logs with logging turned up:

2011-07-19 09:33:25 DEBUG OpenSAML.SecurityPolicyRule.XMLSigning [2]: validating signature profile
2011-07-19 09:33:25 DEBUG XMLTooling.TrustEngine.ExplicitKey [2]: attempting to validate signature with the peer's credentials
2011-07-19 09:33:25 DEBUG XMLTooling.TrustEngine.ExplicitKey [2]: signature validated with credential
2011-07-19 09:33:25 DEBUG OpenSAML.SecurityPolicyRule.XMLSigning [2]: signature verified against message issuer
2011-07-19 09:33:25 DEBUG XMLTooling.TrustEngine.ExplicitKey [2]: attempting to match credentials from peer with end-entity certificate
2011-07-19 09:33:25 DEBUG XMLTooling.TrustEngine.ExplicitKey [2]: end-entity certificate matches peer RSA key information
2011-07-19 09:33:26 ERROR Shibboleth.AttributeResolver.Query [2]: exception during SAML query to https://someidp:8443/idp/profile/SAML2/SOAP/AttributeQuery: CURLSOAPTransport failed while contacting SOAP endpoint (https:// someidp:8443/idp/profile/SAML2/SOAP/AttributeQuery): error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown
2011-07-19 09:33:26 ERROR Shibboleth.AttributeResolver.Query [2]: unable to obtain a SAML response from attribute authority
2011-07-19 09:33:26 INFO Shibboleth.SessionCache [2]: new session created: ID (_dda07d0906181232f9973c0f9068712a) IdP (someidp) Protocol(urn:oasis:names:tc:SAML:2.0:protocol) Address (someaddress)

From my research on this list, it appears that this is the IdP rejecting the SAML signing cert of the SP. I have verfied that the sp-key and sp-cert files on the SP are the right keypair, and that the cert is the one published in the metadata that both these systems (SP and IdP) are using. The metadata is signed with a valid InCommon cert service-issued code signing cert. The IdP nodes and the SP are both using InCommon-issued SSL certs on port 443 ("handle servce" on the IdP), and the IdP uses the SAML cert for SSL on port 8443 ("attribute authority"/SOAP endpoints). This last part (using the SAML signing cert for SSL on port 8443) has struck me as odd in the past, but I get other errors if I use the InCommon-issued SSL cert to protect port 8443. FWIW, we also have IPv6 in the mix end-to-end, and that seems to work fine. In the last log line above, (someaddress) is actually the IPv6 address of the F5 appliance.

Can you give me an idea of why AttributeQuery requests from the SP may be failing? We are using a new DataSealer bean for cryptographic nameIDs, using "cookie persistence" at an F5 BigIP LTM on port 443 to stick SSO sessions to the same IdP node they originated on, and doing "least traffic" load balancing without any persistence on port 8443, relying on the crypto nameID for principal resolution.

Thanks

Nick

------------
Nicholas Roy - Identity Architect
The University of Iowa / ITS - Administrative Information Systems / Directory and Authentication
2800 UCC Iowa City, IA 52242
tel +1 319 335 5510
------------

--
To unsubscribe from this group, send email to
users+un...@shibboleth.net

[Peter Schober]

> 2011-07-19 09:33:26 ERROR Shibboleth.AttributeResolver.Query [2]:
> exception during SAML query to
> https://someidp:8443/idp/profile/SAML2/SOAP/AttributeQuery:
> CURLSOAPTransport failed while contacting SOAP endpoint (https://
> someidp:8443/idp/profile/SAML2/SOAP/AttributeQuery):
> error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate
> unknown

Does the IdP have Apache httpd in front? If so, is the vhost for the
attribute authority properly configured (i.e., "SSLVerifyClient
optional_no_ca")? If running without httpd, did you prepare the
container with the "DelegateToApplication" extension, see (e.g. for
Apache Tomcat)
https://wiki.shibboleth.net/confluence/display/SHIB2/IdPApacheTomcatPrepare

> Can you give me an idea of why AttributeQuery requests from the SP
> may be failing? We are using a new DataSealer bean for
> cryptographic nameIDs, using "cookie persistence" at an F5 BigIP LTM
> on port 443 to stick SSO sessions to the same IdP node they
> originated on, and doing "least traffic" load balancing without any
> persistence on port 8443, relying on the crypto nameID for principal
> resolution.

Attribute queries come directly from the SP to the IdP. The
loadbalancer has no way of knowing that a request from your SP needs
to go to the same IdP node that your HTTP user agent has been routed
to (no cookies, no stickiness). But that's a problem that will only
bite you once you have the SSL layer of the SOAP request running.
-peter

[Peter Schober]

* Peter Schober <peter....@univie.ac.at> [2011-07-19 17:13]:

> > Can you give me an idea of why AttributeQuery requests from the SP
> > may be failing? We are using a new DataSealer bean for
> > cryptographic nameIDs, using "cookie persistence" at an F5 BigIP LTM
> > on port 443 to stick SSO sessions to the same IdP node they
> > originated on, and doing "least traffic" load balancing without any
> > persistence on port 8443, relying on the crypto nameID for principal
> > resolution.

Please disregard the second part of my previous message. If you're in
fact using
https://wiki.shibboleth.net/confluence/display/SHIB2/CryptoTransientPrincipalConnector
querying either IdP node should work, of course. Cookies and session
affility still don't factor into this (but you qualified with "SSO
sessions").
Sorry,

[Roy, Nicholas S]

Thanks Peter-

It looked to me like the AttributeQuery requests were never even getting as far as the IdP software- Tomcat seemed to be rejecting the credentials presented by the client. This makes sense- they are a "self-signed" certificate, and you'd expect the authentication at an IdP level to take place via the signed metadata which contains the certificate for the SP, but since Tomcat knows nothing of the metadata, you'd expect it to reject the self-signed credential. So- I set the Connector definition in the server.xml file for Tomcat from clientAuth="want" to clientAuth="false" - now requests get as far as the IdP software, but since there's no credential present, there's nothing there for the IdP to verify the identity of the SP. Am I doing something obviously wrong? Please forgive me if I missed something obvious.

Thanks,

Nick

[Cantor, Scott E.]

On 7/19/11 1:50 PM, "Roy, Nicholas S" <nichol...@uiowa.edu> wrote:

> So- I set the Connector definition in the server.xml file for Tomcat
>from clientAuth="want" to clientAuth="false" - now requests get as far as
>the IdP software, but since there's no credential present, there's
>nothing there for the IdP to verify the identity of the SP. Am I doing
>something obviously wrong? Please forgive me if I missed something
>obvious.

Take a look at the documentation on prepping your container. All of the
supported containers need to be extended with custom code to handle client
authentication properly.

-- Scott

[Roy, Nicholas S]

We are using the "delegate to application" SSL implementation JAR (http://shibboleth.internet2.edu/downloads/maven2/edu/internet2/middleware/security/tomcat6/tomcat6-dta-ssl/1.0.0/tomcat6-dta-ssl-1.0.0.jar) specified in the documentation: https://wiki.shibboleth.net/confluence/display/SHIB2/IdPApacheTomcatPrepare

The server.xml connector definition for port 8443 looks like this:
<!-- Shibboleth connector added 27 May 2011 - DJK -->
<!-- Attribute Authority SOAP service -->
<Connector port="8443"
protocol="org.apache.coyote.http11.Http11Protocol"
maxHttpHeaderSize="8192"
maxThreads="150"
minSpareThreads="25"
scheme="https"
clientAuth="true"
SSLEnabled="true"
SSLImplementation="edu.internet2.middleware.security.tomcat6.DelegateToApplicationJSSEImplementation"
keystoreFile="E:\Shibboleth\shibboleth-idp\credentials\SAML\idp-test.uiowa.edu SAMLsigning_8443.jks"
keystorePass="removed for my viewing pleasure"
/>

We are using Tomcat 7 as the servlet container. Our Tomcat startup environment contains the following endorsed dirs definition:
-Djava.endorsed.dirs=E:\shibboleth\shibboleth-idp\lib\endorsed

We did not copy the xerces and xalan JARs to the Tomcat\lib directory because when we upgrade Shibboleth, these would then not automatically be upgraded. Is that a mistake?

The documentation leads me to believe that anything newer than Tomcat 6.0.17 should work. The SSL implementation jar seems like it was written for Tomcat 6. Do you think this might be the problem?

Nick

-----Original Message-----
From: Cantor, Scott E. [mailto:cant...@osu.edu]
Sent: Tuesday, July 19, 2011 12:54 PM
To: us...@shibboleth.net
Subject: Re: AttributeQuery failing: CURLSOAPTransport error

[Roy, Nicholas S]

Aha! More information. In the catalina log, I saw this:
Jul 19, 2011 1:26:35 PM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'SSLImplementation' to 'edu.internet2.middleware.security.tomcat6.DelegateToApplicationJSSEImplementation' did not find a matching property.

The Tomcat documentation for Tomcat 7 says that the property for SSL Implementation is now sslImplementationName. I updated this property in my server.xml file, and cycled the container. I now get this in stderr:

java.lang.NoSuchMethodError: org.apache.tomcat.util.net.jsse.JSSESocketFactory: method <init>()V not found
at edu.internet2.middleware.security.tomcat6.DelegateToApplicationJSSEImplementation$NoTrustSocketFactory.<init>(DelegateToApplicationJSSEImplementation.java:73)
at edu.internet2.middleware.security.tomcat6.DelegateToApplicationJSSEImplementation.<init>(DelegateToApplicationJSSEImplementation.java:51)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at java.lang.Class.newInstance0(Unknown Source)
at java.lang.Class.newInstance(Unknown Source)
at org.apache.tomcat.util.net.SSLImplementation.getInstance(SSLImplementation.java:69)
at org.apache.coyote.http11.AbstractHttp11JsseProtocol.init(AbstractHttp11JsseProtocol.java:118)
at org.apache.catalina.connector.Connector.initInternal(Connector.java:910)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:101)
at org.apache.catalina.core.StandardService.initInternal(StandardService.java:559)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:101)
at org.apache.catalina.core.StandardServer.initInternal(StandardServer.java:781)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:101)
at org.apache.catalina.startup.Catalina.load(Catalina.java:572)
at org.apache.catalina.startup.Catalina.load(Catalina.java:595)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:262)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:430)

Nick

[Cantor, Scott E.]

On 7/19/11 2:44 PM, "Roy, Nicholas S" <nichol...@uiowa.edu> wrote:

>The Tomcat documentation for Tomcat 7 says that the property for SSL
>Implementation is now sslImplementationName. I updated this property in
>my server.xml file, and cycled the container. I now get this in stderr:

Then we don't have support yet for Tomcat 7. That's generally always going
to be the case, the "fix the broken SSL connector" code will always break
across releases.

[Roy, Nicholas S]

I can ask the server admin to rebuild our new servers using tomcat 6.latest. Do you know when you might have support for tomcat 7 available? If it's not too far off, we might wait for it since these are new test and production servers that we're building up.

My apologies for missing some earlier comments in this thread. Somehow my junk mail filter got overzealous and shunted some of that into my junk mail folder.

Nick

-----Original Message-----
From: Cantor, Scott E. [mailto:cant...@osu.edu]
Sent: Tuesday, July 19, 2011 2:15 PM
To: us...@shibboleth.net
Subject: Re: AttributeQuery failing: CURLSOAPTransport error

[Cantor, Scott E.]

On 7/19/11 3:19 PM, "Roy, Nicholas S" <nichol...@uiowa.edu> wrote:

>I can ask the server admin to rebuild our new servers using tomcat
>6.latest. Do you know when you might have support for tomcat 7
>available? If it's not too far off, we might wait for it since these are
>new test and production servers that we're building up.

Nobody's actively working on it, and based on what's going on, I doubt
it's going to be imminent unless somebody outside the core team jumps on
it.

Any time I would spend on OSU's dime will be on Jetty at this point. If
nobody else gets to it, it's likely I could take a look at it some time in
August.

At minimum, you'd want to file a RFE, against the IdP I guess.

[Roy, Nicholas S]

Thanks Scott, I've filed an RFE.

Nick

[Roy, Nicholas S]

We've set up Tomcat 6 on these systems, we have the endorsed directory correctly set up which contains the xerces and xalan libraries, we've copied the tomcat6 SSL DTA jar into the tomcat\lib directory, and our server.xml config for port 8443 looks like this:

<Connector port="8443"
protocol="org.apache.coyote.http11.Http11Protocol"
maxHttpHeaderSize="8192"
maxThreads="150"
minSpareThreads="25"
scheme="https"

secure="true"
clientAuth="true"
SSLEnabled="true"
SSLImplemenation="edu.internet2.middleware.security.tomcat6.DelegateToApplicationJSSEImplementation"

keystoreFile="E:\Shibboleth\shibboleth-idp\credentials\SAML\idp-test.uiowa.edu SAMLsigning_8443.jks"

keystorePass="not shown here"
/>

We are still seeing the following message in our test SP logs:

2011-07-20 12:40:58 ERROR Shibboleth.AttributeResolver.Query [6]: exception during SAML query to https://idp-test.uiowa.edu:8443/idp/profile/SAML2/SOAP/AttributeQuery: CURLSOAPTransport failed while contacting SOAP endpoint (https://idp-test.uiowa.edu:8443/idp/profile/SAML2/SOAP/AttributeQuery): error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown
2011-07-20 12:40:58 ERROR Shibboleth.AttributeResolver.Query [6]: unable to obtain a SAML response from attribute authority

It looks like perhaps tomcat is still getting in the way of the IdP handling the SSL on port 8443, but there are no warnings or errors in the tomcat logs. The accesses to this path are not even being logged by tomcat in its access log.

I'm somewhat frustrated- I can't imagine what's happening here at this point. Any guidance is appreciated.

Thanks,

Nick

-----Original Message-----
From: Roy, Nicholas S [mailto:nichol...@uiowa.edu]

Sent: Tuesday, July 19, 2011 2:41 PM
To: us...@shibboleth.net
Subject: Re: AttributeQuery failing: CURLSOAPTransport error

[Cantor, Scott E.]

On 7/20/11 1:57 PM, "Roy, Nicholas S" <nichol...@uiowa.edu> wrote:
>It looks like perhaps tomcat is still getting in the way of the IdP
>handling the SSL on port 8443, but there are no warnings or errors in the
>tomcat logs. The accesses to this path are not even being logged by
>tomcat in its access log.

It is getting in the way. I'm afraid I use Tomcat only with Apache myself.

I think I'd start by sanity checking that it's using the override, maybe
tweak the class name to introduce an error and make sure that fails, that
kind of thing.

[Roy, Nicholas S]

Thanks Scott. It was a PEBKAC error- someone, probably me, had fat-fingered "SSLImplementation" as "SSLImplemenation"

Attributes are now being released by our new test IdP systems via SOAP. Thanks everyone for all your help.

Nick

-----Original Message-----
From: Cantor, Scott E. [mailto:cant...@osu.edu]

Sent: Wednesday, July 20, 2011 1:12 PM
To: us...@shibboleth.net
Subject: Re: AttributeQuery failing: CURLSOAPTransport error