[Shib-Users] metadata problems with newly federated idp
Liam Hoekenga
entityID="https://shibboleth.umich.edu/idp/shibboleth"
to
entityID="urn:mace:incommon:umich.edu"
I've changed this in metadata/idp-metadata.xml. Our test SP fetches
it's metadata directly from the idp, yet when I try to access it, I
get...
2009-05-26 15:19:50 WARN OpenSAML.MessageDecoder.SAML2 [2]: no
metadata found, can't establish identity of issuer
(https://shibboleth.umich.edu/idp/shibboleth)
2009-05-26 15:19:50 WARN Shibboleth.SSO.SAML2 [2]: no metadata found,
can't establish identity of issuer
(https://shibboleth.umich.edu/idp/shibboleth)
2009-05-26 15:19:50 WARN Shibboleth.SSO.SAML2 [2]: detected a problem
with assertion: Unable to establish security of incoming assertion.
I changed the entityID in shibboleth2.xml too, on the SP
<SessionInitiator type="SAML2" Location="/Login" isDefault="true"
defaultACSIndex="1" id="UMich"
entityID="urn:mace:incommon:umich.edu"
template="bindingTemplate.html" />
I thought that I'd read in the mailing list archives that it was a
simple string match between the issuer mentioned in the error message
and the entityID used in the metadata file.
When I go to other federated SPs, I get errors like..
* opensaml::saml2md::MetadataException:
Security of SAML 1.x SSO POST response not established.
* metadata lookup failed, unable to process assertion
This is getting kind of frustrating. Can someone point me in the
right direction?
Liam
Jim Fox
Maybe you forget to change the provider id in your idp's relying-party.xml
file.
Jim
On Tue, 26 May 2009, Liam Hoekenga wrote:
> Date: Tue, 26 May 2009 12:31:36 -0700
> From: Liam Hoekenga <li...@umich.edu>
> To: "shibbole...@internet2.edu" <shibbole...@internet2.edu>
> Reply-To: "shibbole...@internet2.edu" <shibbole...@internet2.edu>
> Subject: [Shib-Users] metadata problems with newly federated idp
Brewer, Edward L
When did you make the change ? They push out changes at 3:00PM
Lee
-----Original Message-----
From: Liam Hoekenga [mailto:li...@umich.edu]
Sent: Tuesday, May 26, 2009 2:32 PM
To: shibbole...@internet2.edu
Subject: [Shib-Users] metadata problems with newly federated idp
Scott Cantor
> So, I registered our idp with incommon. Our entityID went from
> entityID="https://shibboleth.umich.edu/idp/shibboleth"
> to
> entityID="urn:mace:incommon:umich.edu"
Unfortunately.
> I've changed this in metadata/idp-metadata.xml. Our test SP fetches
> it's metadata directly from the idp, yet when I try to access it, I
> get...
Changing it there doesn't change the IdP's configuration, there is no
automatic relationship. You need to fix the IdP's config.
> I changed the entityID in shibboleth2.xml too, on the SP
That's necessary to get the initial request to work once the metadata is
fixed, yes.
> I thought that I'd read in the mailing list archives that it was a
> simple string match between the issuer mentioned in the error message
> and the entityID used in the metadata file.
It is. Your response to the SP has the wrong entityID in it, thus the error.
-- Scott
Liam Hoekenga
Scott Cantor
>
If you get the same log information, then that change didn't take. That's
just a given.
-- Scott
Liam Hoekenga
> If you get the same log information, then that change didn't take. That's
> just a given.
So, I poked around a bit more, and I believe that my entityIDs are all
set correctly; I can log in to my test SP again. I also can get into
federated SP's, but it's not working quite right. It seems to be
working for SAML2 requests (like my SP), but not w/ SAML1 (the
Shibboleth profile handler). I think it's specifically related to the
NameIdentifier encoding.
My IdP is releasing attributes to the SAML1 SPs..
16:48:25.961 - DEBUG
[edu.internet2.middleware.shibboleth.idp.profile.AbstractSAMLProfileHandler:541] - Encoding response to SAML request null from relying party
https://service1.internet2.edu/shibboleth
16:48:25.961 - INFO [Shibboleth-Audit:675] -
20090526T204825Z|urn:mace:shibboleth:1.0:profiles:AuthnRequest||https://service1.internet2.edu/shibboleth|urn:mace:shibboleth:2.0:profiles:saml1:sso|urn:mace:incommon:umich.edu|urn:oasis:names:tc:SAML:1.0:profiles:browser-post|_04710bbb102bcccde59e5113407c5505|liamr|urn:oasis:names:tc:SAML:2.0:ac:classes:PreviousSession||_663efb210895543deef74419143bfda0|_7aa4448afdebfaf153668bb0813b8406,|
It's that "null" that I'm concerned about.
My attribute-resolver.xml looks like..
<!-- Name Identifier related attributes -->
<resolver:AttributeDefinition id="transientId"
xsi:type="TransientId" xmlns="urn:mace:shibboleth:2.0:resolver:ad">
<resolver:AttributeEncoder
xsi:type="SAML1StringNameIdentifier"
xmlns="urn:mace:shibboleth:2.0:attribute:encoder"
nameFormat="urn:mace:shibboleth:1.0:nameIdentifier" />
<resolver:AttributeEncoder xsi:type="SAML2StringNameID"
xmlns="urn:mace:shibboleth:2.0:attribute:encoder"
nameFormat="urn:oasis:names:tc:SAML:2.0:nameid-format:transient" />
</resolver:AttributeDefinition>
W/ my SAML2 SP, I see
16:47:40.835 - DEBUG
[edu.internet2.middleware.shibboleth.idp.profile.saml2.AbstractSAML2ProfileHandler:785] - Using attribute transientId supporting NameID format urn:oasis:names:tc:SAML:2.0:nameid-format:transient to create the
NameID.
...
...
16:47:40.891 - DEBUG
[edu.internet2.middleware.shibboleth.idp.profile.AbstractSAMLProfileHandler:541] - Encoding response to SAML request _d1fc1871cb6a32e3b0e5b8c8a2044b32 from relying party
https://shib-sp-test.www.umich.edu/shibboleth-sp
but with Shibboleth profile handler, I get
16:48:25.961 - DEBUG
[edu.internet2.middleware.shibboleth.idp.profile.saml1.AbstractSAML1ProfileHandler:388] - Using attribute transientId supporting name format urn:mace:shibboleth:1.0:nameIdentifier to create the
NameIdentifier
...
...
16:48:25.961 - DEBUG
[edu.internet2.middleware.shibboleth.idp.profile.AbstractSAMLProfileHandler:541] - Encoding response to SAML request null from relying party
https://service1.internet2.edu/shibboleth
Is it the "null" NameIdentifier that's preventing the SPs from seeing
attributes?
Liam
Scott Cantor
> Is it the "null" NameIdentifier that's preventing the SPs from seeing
> attributes?
I doubt it, it's probably just a logging peculiarity since legacy requests
don't have any identifier to log.
Legacy behavior would presumably involve an attribute query, so if no
attributes are being released despite a filter policy saying to do so, that
would imply plenty of logging available at the SP to indicate why.
-- Scott
Brent Putman
Liam Hoekenga wrote:
>
>
> My IdP is releasing attributes to the SAML1 SPs..
>
> 16:48:25.961 - DEBUG
> [edu.internet2.middleware.shibboleth.idp.profile.AbstractSAMLProfileHandler:541]
> - Encoding response to SAML request null from relying party
> https://service1.internet2.edu/shibboleth
> 16:48:25.961 - INFO [Shibboleth-Audit:675] -
> 20090526T204825Z|urn:mace:shibboleth:1.0:profiles:AuthnRequest||https://service1.internet2.edu/shibboleth|urn:mace:shibboleth:2.0:profiles:saml1:sso|urn:mace:incommon:umich.edu|urn:oasis:names:tc:SAML:1.0:profiles:browser-post|_04710bbb102bcccde59e5113407c5505|liamr|urn:oasis:names:tc:SAML:2.0:ac:classes:PreviousSession||_663efb210895543deef74419143bfda0|_7aa4448afdebfaf153668bb0813b8406,|
>
>
>
> It's that "null" that I'm concerned about.
I think you can safely ignore that. That null piece of data is the SAML
message Id. The Shibboleth profile for SAML 1 doesn't have a SAML
message Id defined. It's a simple query parameter based protocol. That
log message is coming from a component that is agnostic as to SAML
version and protocol message. Arguably we should fix the log message
so it says something more graceful in the case where there is no
protocol message Id. However, it's operationally pretty meaningless
vis-a-vis your problem.
>
>
> Is it the "null" NameIdentifier that's preventing the SPs from seeing
> attributes?
No, it's not related at all, as far as I can tell.
Liam Hoekenga
> Legacy behavior would presumably involve an attribute query, so if no
> attributes are being released despite a filter policy saying to do so, that
> would imply plenty of logging available at the SP to indicate why.
My SP works with my IdP. I don't have access to the logs on the SPs
I'm trying to test against.
Does this log snippet indicate that it's not going to release
attributes to this SP (because it appears to be using the default
relying party configuration)?
10:47:17.160 - DEBUG
[edu.internet2.middleware.shibboleth.common.relyingparty.provider.SAMLMDRelyingPartyConfigurationManager:132] - No custom relying party configuration found for https://spaces.internet2.edu/shibboleth, looking up configuration based on metadata
groups.
10:47:17.160 - DEBUG
[edu.internet2.middleware.shibboleth.common.relyingparty.provider.SAMLMDRelyingPartyConfigurationManager:155] - No custom or group-based relying party configuration found for https://spaces.internet2.edu/shibboleth. Using default relying party
configuration.
10:47:17.160 - DEBUG
[edu.internet2.middleware.shibboleth.idp.profile.saml1.ShibbolethSSOEndpointSelector:78] - Selecting endpoint from metadata corresponding to provided ACS URL:
https://spaces.internet2.edu/Shibboleth.sso/SAML/POST
(I see the same statement about using the default relying party for my
working, SP, though.)
The attributes I'm trying to release are all in a
PolicyRequirementRule that suggests to me that I /should/ be releasing
attributes to InCommon..
<AttributeFilterPolicy>
<PolicyRequirementRule xsi:type="basic:OR">
<basic:Rule
xsi:type="saml:AttributeRequesterInEntityGroup"
groupID="urn:mace:incommon" />
<basic:Rule
xsi:type="saml:AttributeRequesterInEntityGroup"
groupID="http://ukfederation.org.uk" />
<basic:Rule
xsi:type="saml:AttributeRequesterInEntityGroup"
groupID="urn:mace:switch.ch:SWITCHaai" />
<basic:Rule xsi:type="basic:AttributeRequesterString"
value="https://shib-sp-test.www.umich.edu/shibboleth-sp" />
</PolicyRequirementRule>
<AttributeRule attributeID="eduPersonPrincipalName">
<PermitValueRule xsi:type="basic:ANY" />
</AttributeRule>
...cut...
</AttributeFilterPolicy>
I'm guessing that the groupID "urn:mace:incommon" means that it should
consider any SP that's listed in the InCommon metadata file to be part
of that group?
Liam
Scott Cantor
> I'm trying to test against.
You can certainly determine what you're releasing from the IdP, but only
somebody at the SP can fully determine what its passing through and what
specifically might be happening.
You appear to be talking about spaces, which means I can tell you what it's
seeing:
2009-05-27 10:47:17 ERROR XMLTooling.SOAPTransport.CURL [1887]: supplied
TrustEngine failed to validate SSL/TLS server certificate
2009-05-27 10:47:17 ERROR Shibboleth.AttributeResolver.Query [1887]:
exception during SAML query to
https://shibboleth.umich.edu:8443/idp/profile/SAML1/SOAP/AttributeQuery:
CURLSOAPTransport failed while contacting SOAP responder: SSL certificate
problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify
failed
So, your SOAP endpoint is running with an invalid (presumably non-InCommon)
certificate.
> Does this log snippet indicate that it's not going to release
> attributes to this SP (because it appears to be using the default
> relying party configuration)?
Relying party configuration has nothing specifically to do with attribute
release.
> The attributes I'm trying to release are all in a
> PolicyRequirementRule that suggests to me that I /should/ be releasing
> attributes to InCommon..
Possibly, but the query isn't making it through yet, so that's why you don't
see anything.
It's critical when testing that anything you test "locally" rely on the same
metadata as InCommon SPs. This is why it's a mistake to try and separate
"local" from "federated". Once a local system works with the right metadata,
you can be sure anything that doesn't work isn't your problem.
-- Scott
Liam Hoekenga
> So, your SOAP endpoint is running with an invalid (presumably non-InCommon)
> certificate.
*sigh*
This probably would have been easier if I'd just generated a CSR from
the key that shibboleth created when I installed the SP.
Ok, so, yeah, the IdP was using the cert generated during the
installation process (stored in credentials/idp.jks).
I tried adding the inCommon cert to the keystore in addition to the
existing cert.
No go.
I replaced the existing cert (in idp.jks) with the inCommon cert, and
now catalina.out is filled with..
May 27, 2009 11:58:23 AM org.apache.tomcat.util.net.JIoEndpoint$Acceptor run
SEVERE: Socket accept failed
java.net.SocketException: SSL handshake
errorjavax.net.ssl.SSLException: No available certificate or key
corresponds to the SSL cipher suites which are enabled.
at
org.apache.tomcat.util.net.jsse.JSSESocketFactory.acceptSocket(JSSESocketFactory.java:150)
at
org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:310)
at java.lang.Thread.run(Unknown Source)
I see how to get a certificate into a keystore using keytool. How
does one import a key into a keystore?
Liam
Scott Cantor
> the key that shibboleth created when I installed the SP.
Without a doubt, but...
> Ok, so, yeah, the IdP was using the cert generated during the
> installation process (stored in credentials/idp.jks).
I thought you said you were using Apache. I think I mentioned this issue a
while back. If you're using Apache, there's no use for the keystore.
> I see how to get a certificate into a keystore using keytool. How
> does one import a key into a keystore?
Lots of animal sacrifice. We had a tool in the old IdP Walter wrote that can
do it, extkeytool. I think IBM has something lying around that can do it as
well.
But keytool itself definitely cannot do it.
You might also consider, if this is non-Windows, trying out the APR support
in Tomcat, which runs the SSL layer on top of OpenSSL and lets you use the
PEM files you'd like to use.
Or you just run Apache and solve it trivially.
-- Scott
Liam Hoekenga
>> Ok, so, yeah, the IdP was using the cert generated during the
>> installation process (stored in credentials/idp.jks).
>
> I thought you said you were using Apache. I think I mentioned this issue a
> while back. If you're using Apache, there's no use for the keystore.
I *am* using apache, but the keystore is still being used because
tomcat is providing the SOAP endpoint for attribute resolution. The
keystore file it's using has the key that shib generated during the
installation process.
<Connector port="8443"
maxHttpHeaderSize="8192"
maxSpareThreads="75"
scheme="https"
secure="true"
clientAuth="want"
SSLEnabled="true"
sslProtocol="TLS"
keystoreFile="/usr/local/shibboleth-idp/credentials/idp.jks"
keystorePass="***"
truststoreFile="/usr/local/shibboleth-idp/credentials/idp.jks"
truststorePass="***"
truststoreAlgorithm="DelegateToApplication"/>
Should I not use this? Should I publish my attributes over HTTP
instead of SOAP?
Liam
Jim Fox
>
> I *am* using apache, but the keystore is still being used because
> tomcat is providing the SOAP endpoint for attribute resolution. The
> keystore file it's using has the key that shib generated during the
> installation process.
>
This has come up before. Are there instructions somewhere on spaces that
suggest configuring an IdP this way?
Jim
Scott Cantor
> tomcat is providing the SOAP endpoint for attribute resolution.
You should either use Apache or use Tomcat, but not both (as a web server).
> The keystore file it's using has the key that shib generated during the
> installation process.
I understand that, but that connector isn't needed if you're already using
Apache for port 443. Just add a mod_ssl vhost for 8443 with the InCommon
certificate.
> Should I not use this? Should I publish my attributes over HTTP
> instead of SOAP?
SOAP is on top of HTTP. You're already doing HTTP (over SSL) either way. If
you have 443 via Apache and 8443 via Tomcat, that's where the problem
occurred.
-- Scott
Liam Hoekenga
It comes up very quickly in the installation instructions.
Go to Shibboleth installation..
https://spaces.internet2.edu/display/SHIB2/Installation
Follow the Installing the IdP link..
https://spaces.internet2.edu/display/SHIB2/IdPInstall
That page has 3 children. Follow IdPApacheTomcatPrepare..
https://spaces.internet2.edu/display/SHIB2/IdPApacheTomcatPrepare
Liam
Rod Widdowson
> one import a key into a keystore?
Extkeytool (from shib 1).
You might also want to look at
http://tomcat.apache.org/tomcat-6.0-doc/ssl-howto.html
Scott Cantor
> That page has 3 children. Follow IdPApacheTomcatPrepare..
> https://spaces.internet2.edu/display/SHIB2/IdPApacheTomcatPrepare
That's because that set of instructions is for Tomcat ONLY, which we now use
as the default setup because most people do better that way. The Apache
there is because Tomcat is an Apache project, it doesn't refer to the Apache
HTTPD server.
I'm not sure that we have a page for Apache HTTPD + Tomcat at this point, I
guess nobody has gotten around to writing it.
-- Scott
Jim Fox
There is also a pcks12import tool that's part of Sun's web services
developer pack (jwsdp).
Jim
On Wed, 27 May 2009, Rod Widdowson wrote:
> Date: Wed, 27 May 2009 10:02:14 -0700
> From: Rod Widdowson <r...@steadingsoftware.com>
> To: "shibbole...@internet2.edu" <shibbole...@internet2.edu>
> Reply-To: "shibbole...@internet2.edu" <shibbole...@internet2.edu>
> Subject: Re: [Shib-Users] importing keys into a keystore?
Daniel Fisher
>
> I see how to get a certificate into a keystore using keytool. How
> does one import a key into a keystore?
>
> Liam
We distribute a library that you may find useful:
http://code.google.com/p/vt-middleware/wiki/vtcrypt#keystore_-_Keystore_Operations
--Daniel Fisher