I have a concern that 3.2.17 Test Case Q is seriously flawed.
It only tests support for two specific context URN. The current use of this by governments is to express conformance to a specific certification, like the ICAM SAML 2.0 LoA 2 profile.
The relevant section of the SAML spec is 3.3.2.2.1.
The RequestedAuthnContext is a ordered list AuthnContextClassRef each containing a URI. These are compared in order to determine the best match.
As a real example I may send
</saml:AuthnContextClassRef>
</samlp:RequestedAuthnContext>
I expect to get back an assertion that contains one of the two ClassRefs or no AuthnContext.
Working on the catalyst interop it is clear that one or more products that have passed eGov 1.5 can't do this.
Some can only support the test URN.
I would like to see Q expanded to include one or more real URL and sending an ordered list with exact.
This is basic functionality from the core spec that is REQUIRED to meet the ICAM profile.
In testing for catalyst it is also apparent that some venders are only supporting importing single EntityDescriptors from meta-data.
I am quite sure that the requirement is to import EntitiesDescriptor files with multiple EntityDescriptors.
I think Scott may have not been explicit enough, because it is obvious to us that it is a requirement, but perhaps not to others.
I have also discovered some issues where files containing multiple IDPSSODescriptors for the same Entity are not supported.
IOP expects the basic meta-data spec to be supported so I don't think Scott included the super basic tests for meta-data.
I won't start on the security testing:)
John B.