Metadata Aggregator - Issues with XMLSignatureSigningStage
Dan McLaughlin
$ mda.sh file:///Users/Dan/workspace/metadata_aggregator/config1.xml
file:///Users/Dan/workspace/metadata_aggregator/my-test-federation.xml2011-10-22
13:15:05,467 - ERROR
[net.shibboleth.metadata.cli.SimpleCommandLine:70] - Unable to
initialize Spring
contextorg.springframework.beans.factory.BeanCreationException: Error
creating bean with name 'signMetadata' defined in URL
[file:/Users/Dan/workspace/metadata_aggregator/config1.xml]:
Initialization of bean failed; nested exception is
org.springframework.beans.ConversionNotSupportedException: Failed to
convert property value of type 'java.lang.String' to required type
'java.security.PrivateKey' for property 'privateKey'; nested exception
is java.lang.IllegalStateException: Cannot convert value of type
[java.lang.String] to required type [java.security.PrivateKey] for
property 'privateKey': no matching editors or conversion strategy
found at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:527)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:291)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:288)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:190)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:580)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895)
~[spring-context-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425)
~[spring-context-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.context.support.FileSystemXmlApplicationContext.<init>(FileSystemXmlApplicationContext.java:140)
~[spring-context-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.context.support.FileSystemXmlApplicationContext.<init>(FileSystemXmlApplicationContext.java:84)
~[spring-context-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
net.shibboleth.metadata.cli.SimpleCommandLine.main(SimpleCommandLine.java:68)
~[aggregator-cli-0.5-SNAPSHOT.jar:na]Caused by:
org.springframework.beans.ConversionNotSupportedException: Failed to
convert property value of type 'java.lang.String' to required type
'java.security.PrivateKey' for property 'privateKey'; nested exception
is java.lang.IllegalStateException: Cannot convert value of type
[java.lang.String] to required type [java.security.PrivateKey] for
property 'privateKey': no matching editors or conversion strategy
found at org.springframework.beans.BeanWrapperImpl.convertIfNecessary(BeanWrapperImpl.java:462)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.beans.BeanWrapperImpl.convertForProperty(BeanWrapperImpl.java:499)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.beans.BeanWrapperImpl.convertForProperty(BeanWrapperImpl.java:493)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.convertForProperty(AbstractAutowireCapableBeanFactory.java:1371)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1330)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1086)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] ... 11 common frames
omittedCaused by: java.lang.IllegalStateException: Cannot convert
value of type [java.lang.String] to required type
[java.security.PrivateKey] for property 'privateKey': no matching
editors or conversion strategy found at
org.springframework.beans.TypeConverterDelegate.convertIfNecessary(TypeConverterDelegate.java:231)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] at
org.springframework.beans.BeanWrapperImpl.convertIfNecessary(BeanWrapperImpl.java:447)
~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE] ... 17 common frames
omitted
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">
<!-- First, we define the stages for our pipeline -->
<bean id="source"
class="net.shibboleth.metadata.dom.stage.DomFilesystemSourceStage">
<property name="id" value="source"/>
<property name="parserPool">
<bean class="org.opensaml.util.xml.BasicParserPool"
init-method="initialize"/>
</property>
<property name="source">
<bean class="java.io.File">
<constructor-arg
value="/Users/Dan/workspace/metadata_aggregator/entities"/>
</bean>
</property>
<property name="recurseDirectories" value="true"/>
</bean>
<bean id="removeContactPerson"
class="net.shibboleth.metadata.dom.saml.RemoveContactPersonStage">
<property name="id" value="removeContactPerson"/>
</bean>
<bean id="removeOrganization"
class="net.shibboleth.metadata.dom.saml.RemoveOrganizationStage">
<property name="id" value="removeOrganization"/>
</bean>
<bean id="createEntitiesDescriptor"
class="net.shibboleth.metadata.dom.saml.EntitiesDescriptorAssemblerStage">
<property name="id" value="createEntitiesDescriptor"/>
</bean>
<bean id="signMetadata"
class="net.shibboleth.metadata.dom.stage.XMLSignatureSigningStage">
<property name="id" value="signMetadata"/>
<property name="privateKey"
value="/Users/Dan/workspace/metadata_aggregator/sp-key.pem"/>
</bean>
<!-- Next we define a pipeline with all the stages in it -->
<bean id="pipeline" class="net.shibboleth.metadata.pipeline.SimplePipeline">
<property name="id" value="pipeline"/>
<property name="stages">
<list>
<ref bean="source"/>
<ref bean="removeContactPerson"/>
<ref bean="removeOrganization"/>
<ref bean="createEntitiesDescriptor"/>
<ref bean="signMetadata"/>
</list>
</property>
</bean>
<!-- Lastly we define a serializer that can write out our metadata -->
<bean id="serializer"
class="net.shibboleth.metadata.dom.DomMetadataSerializer"/>
</beans>
--
Thanks,
Dan McLaughlin
--
To unsubscribe from this list send an email to dev-uns...@shibboleth.net
Brent Putman
On 10/22/11 2:24 PM, Dan McLaughlin wrote:
> Initialization of bean failed; nested exception is
> org.springframework.beans.ConversionNotSupportedException: Failed to
> convert property value of type 'java.lang.String' to required type
> 'java.security.PrivateKey' for property 'privateKey'; nested exception
> is java.lang.IllegalStateException: Cannot convert value of type
> [java.lang.String] to required type [java.security.PrivateKey] for
> property 'privateKey':
I haven't used it, but the error is pretty clear: you're trying to
inject a String where you need to inject a PrivateKey instance.
> <bean id="signMetadata"
> class="net.shibboleth.metadata.dom.stage.XMLSignatureSigningStage">
> <property name="id" value="signMetadata"/>
> <property name="privateKey"
> value="/Users/Dan/workspace/metadata_aggregator/sp-key.pem"/>
> </bean>
>
The error is here. You can't inject the path to a PEM-encoded file
there, you need to inject the actual PrivateKey instance.
I know Chad has a Spring extension helper FactoryBean for that, perhaps
you should be using that to wire the PrivateKey key from a java.io.File
and then inject the bean ref to the PrivateKey.
Dan McLaughlin
from the EDS example here...
I'll take a look at the link you sent. Thanks!
--
Thanks,
Dan McLaughlin
Brent Putman
code is under heavy development, so probably those examples have just
fallen out-of-sync with the code. Probably Chad or Ian can clarify the
latest state of things.
Ian Young
On 22 Oct 2011, at 22:24, Brent Putman wrote:
> Looks like those config examples were added in April 2011. I know this
> code is under heavy development, so probably those examples have just
> fallen out-of-sync with the code. Probably Chad or Ian can clarify the
> latest state of things.
The docs on the wiki (including the examples but also the JavaDoc) appear to date to version 0.5. There was a *lot* of refactoring for 0.6, and those examples definitely won't work on many levels. Classes have moved around, properties have changed, even the command-line parameters have changed.
We should obviously fix what's in the wiki, but it might take a while. For reference material at the moment I personally have a copy of the sources in Eclipse projects pulled from the repository. I realise that's not very end-user-friendly!
If people need examples to look at, I can come up with quite a lot of complex ones (as we're running the whole UK federation metadata build off the 0.6 codebase) but unfortunately I'm a bit short of simple ones.
Dan: let me know how far you've got since you initially posted, and I'll see if I can't get you something that's a bit more useful. We definitely want more people poking at this if possible.
-- Ian
Dan McLaughlin
verification, the rest of the examples from the wiki are working with
0.5.
--
Thanks,
Dan McLaughlin
Peter Schober
> If people need examples to look at, I can come up with quite a lot
> of complex ones (as we're running the whole UK federation metadata
> build off the 0.6 codebase) but unfortunately I'm a bit short of
> simple ones.
Examples would be nice. Not sure when I'll get around to actually
looking at them, though. Only learned of the existence of that tool
from Lukas last week.
-peter
Chad La Joie
As Ian mentioned, there is a semi-released 0.6 that changes some
things. I say semi-released because there was a compiler issue found
and I was going to fix it before release but then got side tracked by
this summer's security issues and then IdP releases.
So, give me a couple days to fix up the compiler issue and update the
wiki and then test with that. As Brent mentioned there are some
helper classes that I did that make it easier to create keys and certs
and the like so that should make what you're doing easier.
On Thu, Oct 27, 2011 at 17:11, Dan McLaughlin
<dmcla...@tech-consortium.com> wrote:
> The only ones that I need examples for are signing and schema
> verification, the rest of the examples from the wiki are working with
> 0.5.
--
Chad La Joie
www.itumi.biz
trusted identities, delivered
Chad La Joie
As far as not knowing about it, not sure what to tell you. It was
announced on the mailing list. Nicole suggested we send something to
the refeds list and I'll probably do that once I get 0.6 officially
released.
On Thu, Oct 27, 2011 at 17:13, Peter Schober <peter....@univie.ac.at> wrote:
> Examples would be nice. Not sure when I'll get around to actually
> looking at them, though. Only learned of the existence of that tool
> from Lukas last week.
--
Chad La Joie
www.itumi.biz
trusted identities, delivered
Peter Schober
> As far as not knowing about it, not sure what to tell you. It was
> announced on the mailing list. Nicole suggested we send something to
> the refeds list and I'll probably do that once I get 0.6 officially
> released.
I wasn't complaining and indeed there were two messages about this to
the dev list, one in Sept 2010 and one in April this year.
I glanced over the MA1 wiki during a conference and got lost on the
General Configuration page. Will certainly revisit.
-peter
Chad La Joie
aggregator is not for the faint of heart. You have to know about
Spring and you have to understand metadata and your needs pretty
clearly before trying to do much with it.
--
Chad La Joie
www.itumi.biz
trusted identities, delivered
Chad La Joie
I just released 0.6.1 and made sure all the documentation was update.
I also include a new "non-trivial" example that demonstrates a lot of
the stages. So, give that a whirl.
--
Chad La Joie
www.itumi.biz
trusted identities, delivered
Krug, Jeff
I did have one question regarding signing algorithm. Using the xmlsectool-1.1.4 I tweaked it to default to SHA256 signatures (and it uses Apache's digital signature classes to do this). This worked fine. The aggregator defaults to SHA256 (although conveniently configurable via a property) using the javax.crypto libraries, but for this I get the following error:
2011-11-02 10:52:30,398 - ERROR [net.shibboleth.metadata.dom.XMLSignatureSigningStage:644] - Unable to create signature method http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
java.security.NoSuchAlgorithmException: unsupported algorithm
at org.jcp.xml.dsig.internal.dom.DOMXMLSignatureFactory.newSignatureMethod(Unknown Source) ~[na:1.6.0_16]
at net.shibboleth.metadata.dom.XMLSignatureSigningStage.buildSignedInfo(XMLSignatureSigningStage.java:641) [aggregator-pipeline-0.6.1.jar:na]
I can set it to use SHA1 via the property and it works fine, but I feel like there is something obvious I'm overlooking that needs to be done to support SHA256 (and better, the same type of error shows up for SHA384 and SHA512).
Thanks,
Jeff
________________________________________
From: dev-b...@shibboleth.net [dev-b...@shibboleth.net] on behalf of Chad La Joie [laj...@itumi.biz]
Sent: Sunday, October 30, 2011 2:45 PM
To: Shib Dev
Subject: Re: Metadata Aggregator - Issues with XMLSignatureSigningStage
Chad La Joie
Java XML DSIG APIs. My initial guess is that the implementation that
ships with the JVM doesn't support anything but the what is explicitly
defined in the DSIG spec. I'll dig in to it a bit.
On Wed, Nov 2, 2011 at 11:04, Krug, Jeff <Jeff...@gtri.gatech.edu> wrote:
> I did have one question regarding signing algorithm. Using the xmlsectool-1.1.4 I tweaked it to default to SHA256 signatures (and it uses Apache's digital signature classes to do this). This worked fine. The aggregator defaults to SHA256 (although conveniently configurable via a property) using the javax.crypto libraries, but for this I get the following error:
>
> 2011-11-02 10:52:30,398 - ERROR [net.shibboleth.metadata.dom.XMLSignatureSigningStage:644] - Unable to create signature method http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
> java.security.NoSuchAlgorithmException: unsupported algorithm
> at org.jcp.xml.dsig.internal.dom.DOMXMLSignatureFactory.newSignatureMethod(Unknown Source) ~[na:1.6.0_16]
> at net.shibboleth.metadata.dom.XMLSignatureSigningStage.buildSignedInfo(XMLSignatureSigningStage.java:641) [aggregator-pipeline-0.6.1.jar:na]
>
> I can set it to use SHA1 via the property and it works fine, but I feel like there is something obvious I'm overlooking that needs to be done to support SHA256 (and better, the same type of error shows up for SHA384 and SHA512).
--
Tom Poage
On Nov 2, 2011, at 8:04 AM, Krug, Jeff wrote:
> I did have one question regarding signing algorithm. Using the xmlsectool-1.1.4 I tweaked it to default to SHA256 signatures (and it uses Apache's digital signature classes to do this). This worked fine. The aggregator defaults to SHA256 (although conveniently configurable via a property) using the javax.crypto libraries, but for this I get the following error:
>
> 2011-11-02 10:52:30,398 - ERROR [net.shibboleth.metadata.dom.XMLSignatureSigningStage:644] - Unable to create signature method http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
> java.security.NoSuchAlgorithmException: unsupported algorithm
> at org.jcp.xml.dsig.internal.dom.DOMXMLSignatureFactory.newSignatureMethod(Unknown Source) ~[na:1.6.0_16]
> at net.shibboleth.metadata.dom.XMLSignatureSigningStage.buildSignedInfo(XMLSignatureSigningStage.java:641) [aggregator-pipeline-0.6.1.jar:na]
>
> I can set it to use SHA1 via the property and it works fine, but I feel like there is something obvious I'm overlooking that needs to be done to support SHA256 (and better, the same type of error shows up for SHA384 and SHA512).
--
Krug, Jeff
________________________________________
From: dev-b...@shibboleth.net [dev-b...@shibboleth.net] on behalf of Tom Poage [tfp...@ucdavis.edu]
Sent: Wednesday, November 02, 2011 11:29 AM
To: Shib Dev
Subject: Re: Metadata Aggregator - Issues with XMLSignatureSigningStage
Missing JCE Unlimited Strength Policy files, perhaps?
Chad La Joie
--
Chad La Joie
www.itumi.biz
trusted identities, delivered
Chad La Joie
supported (I can never find those lists, Brent probably knows where
they are).
So, there's a fairly easy way to test this. Endorse the santuario
library. Just copy the xmlsec-1.4.5.jar in to your endorsed directory
where the xerces and xalan jars should be. Then try again.
Krug, Jeff
2011-11-02 11:50:16,271 - ERROR [net.shibboleth.metadata.cli.SimpleCommandLine:68] - Unable to initialize Spring context
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'main' defined in URL [file:/home/jk90/src/aggregator-cli-0.6.1/config1.xml]: Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1420) ~[spring-beans-3.0.5.RELEASE.jar:3.0.5.RELEASE]
________________________________________
From: dev-b...@shibboleth.net [dev-b...@shibboleth.net] on behalf of Chad La Joie [laj...@itumi.biz]
Sent: Wednesday, November 02, 2011 11:46 AM
To: Shib Dev
Subject: Re: Metadata Aggregator - Issues with XMLSignatureSigningStage
Okay, I wasn't able to find the list of what algos are actually
Chad La Joie
While the xmlsec library does implement the JSR 105 APIs, it makes the
assumption Apache's logging libraries are going to be present, which
obviously they aren't within the JVM's system classloader. I'll need
to talk with them about it. We may be ale to do something within our
code to force the xmlsec JSR105 implementation, but I'd rather not do
that. Kind of defeats the purpose of using a standard API.
Krug, Jeff
Thanks,
Jeff
________________________________________
From: dev-b...@shibboleth.net [dev-b...@shibboleth.net] on behalf of Krug, Jeff [Jeff...@gtri.gatech.edu]
Sent: Wednesday, November 02, 2011 11:52 AM
To: Shib Dev
Subject: RE: Metadata Aggregator - Issues with XMLSignatureSigningStage
Chad La Joie
On Wed, Nov 2, 2011 at 12:31, Krug, Jeff <Jeff...@gtri.gatech.edu> wrote:
> As one other note, there is a typo on line 168 of the config3.xml example. It should be readLocalMetadata instead of readUkMetadata... Although that doesn't really reduce it's usefulness as an example!
--
Chad La Joie
You can endorse xmlsec but you also have to add the Apache commons
logging jar[1] to the endorsed directory as well. I've tried this and
it works okay and does support SHA256 and other algos. I do have some
reservations about the full impact of putting "random" jars in the
endorsed directory but for a command line tool at least it'll be fine.
[1] http://commons.apache.org/logging/download_logging.cgi
Krug, Jeff
I updated config3.xml on the wiki. I did not realize attachments were fair game to update as well.
Thanks,
Jeff
________________________________________
From: dev-b...@shibboleth.net [dev-b...@shibboleth.net] on behalf of Chad La Joie [laj...@itumi.biz]
Sent: Wednesday, November 02, 2011 1:08 PM
Brent Putman
On 11/2/11 11:46 AM, Chad La Joie wrote:
> Okay, I wasn't able to find the list of what algos are actually
> supported (I can never find those lists, Brent probably knows where
> they are).
>
Well, the "regular" (i.e non JSR 105) Santuario config file, where they
have all of the algorithm mapping stuff, is here:
org/apache/xml/security/resource/config.xml
However, looking at revisions in svn that file has had the rsa-sha256
support since at least 2002, so that's not the problem.
I looked at the JSR 105 classes, the factory class below referenced in
the error actually hardcodes the supported algorithm URI -> class impl
mappings, and it's the thing throwing the NoSuchAlgorithmException below:
org.jcp.xml.dsig.internal.dom.DOMXMLSignatureFactory.newSignatureMethod
It didn't add the rsa-sha256 support until 4/11/08 according to svn. So
must have been one of the earlier versions of xmlsec that made it into
Oracle Java 6.
>>> 2011-11-02 10:52:30,398 - ERROR [net.shibboleth.metadata.dom.XMLSignatureSigningStage:644] - Unable to create signature method http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
>>> java.security.NoSuchAlgorithmException: unsupported algorithm
>>> at org.jcp.xml.dsig.internal.dom.DOMXMLSignatureFactory.newSignatureMethod(Unknown Source) ~[na:1.6.0_16]
>>> at net.shibboleth.metadata.dom.XMLSignatureSigningStage.buildSignedInfo(XMLSignatureSigningStage.java:641) [aggregator-pipeline-0.6.1.jar:na]
>
Dan McLaughlin
I have run into an issue if I create subdirectories under
/tmp/mda/entities with metadata files, then DomFilesystemSourceStage
gets stuck in a recursive loop. See exception below...
<bean id="source"
class="net.shibboleth.metadata.dom.DomFilesystemSourceStage"
p:id="source">
<property name="parserPool">
<bean class="org.opensaml.util.xml.BasicParserPool"/>
</property>
<property name="source">
<bean class="java.io.File">
<constructor-arg value="/tmp/mda/entities"/>
</bean>
</property>
<property name="recurseDirectories" value="true"/>
</bean>
Exception in thread "main" java.lang.StackOverflowError
at sun.nio.cs.UTF_8$Encoder.encodeLoop(UTF_8.java:447)
at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:544)
at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:240)
at java.lang.StringCoding.encode(StringCoding.java:272)
at java.lang.String.getBytes(String.java:946)
at java.io.UnixFileSystem.list(Native Method)
at java.io.File.list(File.java:973)
at java.io.File.listFiles(File.java:1051)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:250)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.dom.DomFilesystemSourceStage.getSourceFiles(DomFilesystemSourceStage.java:254)
at net.shibboleth.metadata.d
--
Thanks,
Dan McLaughlin
Chad La Joie
James Webb
I'm working with Dan here on this project. With a little but of debugging, I found the issue with the recursion in DomFilesystemSourceStage.getSourceFiles.
// file must be a directory
- final File[] files = sourceFile.listFiles();
+ final File[] files = input.listFiles();
if (files != null) {
for (File file : files) {
if (file.isFile() || (file.isDirectory() && recurseDirectories)) {
Essentially, the code was in an infinite loop as soon as it hit its first directory.
I'm attaching a patch for that change. We'll file a bug and include the patch in there.
Thanks again for putting together this awesome tool!
Regards,
James Webb
________________________________________
From: dev-b...@shibboleth.net [dev-b...@shibboleth.net] on behalf of Chad La Joie [laj...@itumi.biz]
Sent: Friday, November 04, 2011 4:57 PM
To: Shib Dev
Subject: Re: Metadata Aggregator - Issues with XMLSignatureSigningStage
Probably a bug, go ahead and file it. I'm away on vacation until Nov 21.
Dan McLaughlin
Bug filed. https://issues.shibboleth.net/jira/browse/MDA-53
I tested the patch provided by James and it fixed the issue.
Enjoy your vacation!
--
Thanks,
Dan McLaughlin
Dan McLaughlin
tried to get our SP to consume the metadata it aggregated and signed,
but every time I enabled the Signature MetadataFilter to validate the
signature I would get an error telling me "CRIT Shibboleth.Application
: error initializing MetadataProvider: SignatureMetadataFilter unable
to verify signature at root of metadata instance."
I assumed maybe my private/public key pair I was using to sign and
validate the metadata was bad, so I used openssl to verify that the
private key I used with the MDA to sign the metadata matched the
public key I was using in the SP to validate the signature. Long
story short, openssl confirmed they matched.
Then I used xmlsectool to validate the signature on the metadata
generated by MDA and it complained as well, but gave me a little more
detail.
xmlsectool.sh --verifySignature --certificate
./certs/my-signing-cert.pem --inFile
/tmp/mda/federation/my-federation-metadata.xml
INFO XmlSecTool - Reading XML document from file
'/tmp/mda/federation/my-federation-metadata.xml'
INFO XmlSecTool - XML document parsed and is well-formed.
ERROR XmlSecTool - Unknown error
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.String.substring(String.java:1937) ~[na:1.6.0_29]
at java.lang.String.substring(String.java:1904) ~[na:1.6.0_29]
at edu.internet2.middleware.security.XmlSecTool.validateSignatureReferenceUri(XmlSecTool.java:623)
~[xmlsectool-1.1.5.jar:na]
at edu.internet2.middleware.security.XmlSecTool.validateSignatureReference(XmlSecTool.java:602)
~[xmlsectool-1.1.5.jar:na]
at edu.internet2.middleware.security.XmlSecTool.verifySignature(XmlSecTool.java:554)
~[xmlsectool-1.1.5.jar:na]
at edu.internet2.middleware.security.XmlSecTool.main(XmlSecTool.java:156)
~[xmlsectool-1.1.5.jar:na]
Which led me to
https://issues.shibboleth.net/jira/browse/XSTJ-15?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#issue-tabs
Then I decided to add an XMLSignatureValidationStage to my MDA
configuration to validate the signature after the MDA signed it, and I
got the following error:
mda.sh /tmp/mda/my-federation-config.xml validateSignature
2012-01-23 19:39:46,391 - WARN
[org.apache.xml.security.signature.Reference:-1] - Verification failed
for URI ""
2012-01-23 19:39:46,394 - WARN
[org.apache.xml.security.signature.Reference:-1] - Expected Digest:
I0I+qxu89yE2c6grAFkgO+IbgaEv9DIhCYiGe+JDA/Q=
2012-01-23 19:39:46,395 - WARN
[org.apache.xml.security.signature.Reference:-1] - Actual Digest:
usLmjOJIFNbagPGDEVXW0C0fhwLEtZ8jt0FWWx0/VIA=
As I test I manually added an ID (ID=MYM20120123T194212) to the
EntitiesDescriptor that the MDA created using the
EntitiesDescriptorAssemblerStage, then I used xmlsectool to sign the
metadata using --referenceIdAttributeName ID. Now I had no issues
validating the signature.
./xmlsectool.sh --sign --referenceIdAttributeName ID --inFile
/tmp/mda/federation/my-federation-metadata-unsigned.xml --key
../certs/my-signing-key.pem --certificate ../certs/my-signing-cert.pem
--outFile /tmp/mda/federation/my-federation-metadata.xml
INFO XmlSecTool - Reading XML document from file
'/tmp/mda/federation/my-federation-metadata-unsigned.xml'
INFO XmlSecTool - XML document parsed and is well-formed.
INFO XmlSecTool - XML document successfully signed
INFO XmlSecTool - XML document written to file
/tmp/mda/federation/my-federation-metadata.xml
./xmlsectool.sh --verifySignature --signatureRequired --certificate
../certs/my-signing-cert.pem --inFile
/tmp/mda/federation/my-federation-metadata.xml
INFO XmlSecTool - Reading XML document from file
'/tmp/mda/federation/my-federation-metadata.xml'
INFO XmlSecTool - XML document parsed and is well-formed.
INFO XmlSecTool - XML document signature verified.
I think I'm on the right track...
The signature that the MDA is adding isn't valid because the Reference
URI for the Signature isn't getting set by my
XMLSignatureSigningStage, the reason the Reference URI isn't getting
set is because the EntitiesDescriptorAssemblerStage doesn't set the ID
for the EntitiesDescriptor, the reason the ID isn't getting set for
the EntitiesDescriptor is because I don't have a
EntityDescriptorItemIdPopulationStage, and the reason I don't have an
EntityDescriptorItemIdPopulationStage is because I used the examples
to build my MDA configuration (which also don't use an
EntityDescriptorItemIdPopulationStage), so I never realized until
after several hours of debugging today that it was even necessary.
Now for the difficult question...Does anyone have an example that
shows how to properly define an EntityDescriptorItemIdPopulationStage
so I can get an ID assigned to my EntitiesDescriptor?
Chad La Joie
EntityDescriptorItemIdPopulationStage is meant for something else[1].
So, for now, you should do two things. First, file a bug. Second,
create a stage that generates and sets an ID on an EntityDescriptor or
EntitiesDescriptor (which is what I'll do when I fix the issue).
[1] This stage is meant to take the identifier of an EntityDescriptor
and add it to the metadata of the given item. That is, it takes a
SAML specific thing and copies it to a protocol agnostic location so
that other plugins can work on it. Once the MDA web service is
available, this is also how it will look up information by ID.
--
Chad La Joie
www.itumi.biz
trusted identities, delivered
Dan McLaughlin
Is there an application requirements document, or even better, a design document for the MDA that you could share? If we were to spend the time to add the code to the MDA to fix this, I'd want to make sure we were doing it only if we were writing to and existing MDA design document. In other words, there's no since in us writing a fix to create this stage to add the ID if it's not how you had intended it to be written.
--
Thanks,
Dan McLaughlin
Technology Consortium, LLC
dmcla...@tech-consortium.com
mobile: 512.633.8086
http://www.tech-consortium.com
NOTICE: This e-mail message and all attachments transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is strictly prohibited. The contents of this e-mail are confidential and may be subject to work product privileges. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.
Need to schedule a meeting??? http://www.tungle.me/DanMcLaughlin
Chad La Joie
includes the general architecture. Given that this is a general
processing pipeline, there is no right or wrong stage. There are only
stages that either meet your particular needs and those that don't.
When I fix this bug, I'm going to create a stage that generates a
random ID. Other users, I know, have particular ID generation
algorithms that use proprietary information to generate the ID.
Neither approach is right or wrong, just one meets some needs and the
other meets other needs.
Dan McLaughlin
On a related note... I noticed this morning that every time the SP
downloads the metadata I generated using the MDA, the backingfile has
been modified by the SP to add a standalone="no" declaration to the
XML header, it has also modified every EntityDescriptor element so its
entityID attribute is listed after the attribute
xmlns="urn:oasis:names:tc:SAML:2.0:metadata".
This leads me to ask two questions...
1) Doesn't the fact that the SP is modifying the metadata before
placing it in the backingfile invalidate the signature? May it
doesn't matter once the SP has consumed it, but I'm still curious.
2) Is the SP modifying the metadata only because the MDA didn't
generate it properly to begin with?
Chad La Joie
<dmcla...@tech-consortium.com> wrote:
> Is this something you plan to fix soon? If so, I'll wait to test it.
No, it'll be at least a few more weeks before I switch back to working
on the aggregator.
> 1) Doesn't the fact that the SP is modifying the metadata before
> placing it in the backingfile invalidate the signature? May it
> doesn't matter once the SP has consumed it, but I'm still curious.
Depends on what is being modified. If it's outside the signature,
then no it won't affect anything.
> 2) Is the SP modifying the metadata only because the MDA didn't
> generate it properly to begin with?
The MDA will generate invalid XML in some cases (there are issues for
those in the wiki), but in those cases the SP would fail to accept it
at all. You'd have to ask Scott exactly what the SP does, with
metadata it accepts prior to writing it to its backup file. Chance
are he isn't reading this thread since he doesn't have anything to do
with the MDA (nor is this an MDA question).
Cantor, Scott
> downloads the metadata I generated using the MDA, the backingfile has
> been modified by the SP to add a standalone="no" declaration to the
> XML header, it has also modified every EntityDescriptor element so its
> entityID attribute is listed after the attribute
> xmlns="urn:oasis:names:tc:SAML:2.0:metadata".
Order of attributes is not significant in XML.
> This leads me to ask two questions...
>
> 1) Doesn't the fact that the SP is modifying the metadata before
> placing it in the backingfile invalidate the signature? May it
> doesn't matter once the SP has consumed it, but I'm still curious.
It's not invalid.
> 2) Is the SP modifying the metadata only because the MDA didn't
> generate it properly to begin with?
It's not modifying it, it's serializing it. That's not under my control, Xerces is responsible for that step. The original DOM is not modified, and there are no unusual settings used. The file is not archived as a usable backup until the filtering step completes, so an invalid signature will never be backed up (that used to happen).
-- Scott
Dan McLaughlin
>
> No, it'll be at least a few more weeks before I switch back to working
> on the aggregator.
Okay, we are going to look into coding something up today. If you
have any thoughts on how you would like it to work, naming
conventions, etc... let me know now. I'm assuming you would want the
stage designed in such a way that the method used to generate the ID
was pluggable. I haven't looked at the code yet, but would you
expect the Reference URI to get populated properly by the existing
XMLSignatureSigningStage code once the ID has been added to the
EntitiesDescriptor, or do you expect will have to modify the
XMLSignatureSigningStage to have an option similar to the xmlsectool
referenceIdAttributeName option?
>
>> 1) Doesn't the fact that the SP is modifying the metadata before
>> placing it in the backingfile invalidate the signature? May it
>> doesn't matter once the SP has consumed it, but I'm still curious.
>
> Depends on what is being modified. If it's outside the signature,
> then no it won't affect anything.
After looking over
(http://www.ibm.com/developerworks/webservices/library/ws-security.html)
it would seem due to the CanonicalizationMethod applied to the
signature that the changes made by the SP will not invalidate the
signature.
>
>> 2) Is the SP modifying the metadata only because the MDA didn't
>> generate it properly to begin with?
>
> The MDA will generate invalid XML in some cases (there are issues for
> those in the wiki), but in those cases the SP would fail to accept it
> at all. You'd have to ask Scott exactly what the SP does, with
> metadata it accepts prior to writing it to its backup file. Chance
> are he isn't reading this thread since he doesn't have anything to do
> with the MDA (nor is this an MDA question).
Since the serialization of the metadata by the SP/Xerces doesn't
invalidate the signature, then I'm not sure it matters.