SSP SP supporting multiple IDPs via discovery and MDQ

[Liam Hoekenga]

Here's my use case.... I need to set up an SP using SSP.  This SP is for a research collaboration between several dozen higher ed institutions in the US.  

I would like to configure this SP to use a discovery service to allow users to select their home institutions.  The IDPs are all registered with InCommon, so I thought I'd use the InCommon MDQ server to retrieve the IDP metadata.

I think that the discovery service options (idpdisco.*) in config.php are all intended for the legacy discovery service (modules/saml/www/disco.php) which does not seem to be a part of the SSP 2.x distribution.

Is it possible to configure SSP w/ a list of IDP entities and a pointer to the MDQ server, so I don't have to set up individual IDP entries in saml20-idp-remote.php?

Can that list then be used with the new discovery service (discopower?)?

Any pointers or sample configuration would be very much appreciated.

Liam

[Dick Visser]

What does the relevant configuration (metadata.sources in config.php) look like?

--
This is a mailing list for users of SimpleSAMLphp, not a support service. If you are willing to buy commercial support, please take a look here:
 
https://simplesamlphp.org/support
 
Before sending your question, make sure it is related to SimpleSAMLphp, and not your web server's configuration or any other third-party software. This mailing list cannot help with software that uses SimpleSAMLphp, only regarding SimpleSAMLphp itself.
 
Make sure to read the documentation:
 
https://simplesamlphp.org/docs/stable/
 
If you have an issue with SimpleSAMLphp that you cannot resolve and reading the documentation doesn't help, you are more than welcome to ask here for help. Subscribe to the list and send an email with your question. However, you will be expected to comply with some minimum, common sense standards in your questions. Please read this carefully:
 
http://catb.org/~esr/faqs/smart-questions.html
---
You received this message because you are subscribed to the Google Groups "SimpleSAMLphp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to simplesamlph...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/simplesamlphp/3f2732e0-6eef-4b92-b656-ff4cf64e0f5dn%40googlegroups.com.

[Liam Hoekenga]

I don't know if I mentioned this - I'm using SSP 2.2.2 w/ PHP 8.3.8 and Apache HTTPd 2.4.59

On Tue, Jul 2, 2024 at 11:21 AM Dick Visser <dnmv...@gmail.com> wrote:

What does the relevant configuration (metadata.sources in config.php) look like?

    'metadata.sources' => [
        [
            'type' => 'mdq',
            'server' => 'https://mdq.incommon.org',
            'validateCertificate' => [
                '/opt/local/simplesamlphp/cert/inc-md-cert-mdq.pem'
            ],
            'cachedir' => '/opt/local/var/cache/simplesamlphp/mdq',
            'cachelength' => 86400
        ],

    ], 

ChatGPT also suggested this piece of configuration for saml20-idp-remote.php, but I don't know if it's a hallucination...

 $metadata['default'] = array(

    'mdq' => true,

    'mdq.server' => 'https://mdq.incommon.org/',

    'mdq.cache.duration' => 86400, // Cache duration in seconds

);

If I set a specific IDP for the SP in authsources.php, e.g. 

$config['default-sp']['idp'] = 'https://idp.example.edu/';

it does look it up via MDQ.  If I leave idp set to null and set the discoURL, I do get directed to a discovery page, but it's empty.

And I see this in my logs..
Jul 1 15:05:38 simplesamlphp INFO [4791e38f97] PowerIdPDisco.poweridpdisco: Accessing discovery service.
Jul 1 15:05:38 simplesamlphp INFO [4791e38f97] PowerIdPDisco.poweridpdisco: returnIdParam initially set to [idpentityid]
Jul 1 15:05:38 simplesamlphp INFO [4791e38f97] PowerIdPDisco.poweridpdisco: isPassive initially set to [FALSE]
Jul 1 15:05:38 simplesamlphp INFO [4791e38f97] PowerIdPDisco.poweridpdisco: getSelectedIdP() returned null

and then it asks the MDQ server for my SSP's metadata (which seems like weird behavior to me)...
Jul 1 15:05:38 simplesamlphp INFO [4791e38f97] SimpleSAML\Metadata\Sources\MDQ: loading metadata entity [https://testsp.example.edu/ssp/] from [saml20-sp-remote]
Jul 1 15:05:38 simplesamlphp DEBUG [4791e38f97] SimpleSAML\Metadata\Sources\MDQ: downloading metadata for "https://testsp.example.edu/ssp/" from [https://mdq.incommon.org/entities/https%3A%2F%2Ftestsp.example.edu%2Fssp%2F]
Jul 1 15:05:39 simplesamlphp INFO [4791e38f97] Unable to fetch metadata for "https://testsp.example.edu/ssp/" from https://mdq.incommon.org/entities/https%3A%2F%2Ftestsp.example.edu%2Fssp%2F: file_get_contents(https://mdq.incommon.org/entities/https%3A%2F%2Ftestsp.example.edu%2Fssp%2F): Failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found

It's not clear to me where discopower gets the list of IDPs to populate the discovery service.  It looks like the v1.xx disco.php used $config['idpdisco.external']['layout']['links'] ?

    'idpdisco.external' => [
        'mdq' => [
            'class' => 'disco:MDQ',
            'url' => 'https://mdq.incommon.org/entities/',
        ],
        'layout' => [
            'use' => 'links',
            'links' => [
                'https://idp.example.edu',

                // Add more IdP entity IDs as needed
                'https://idp.educause.edu/idp/shibboleth',
                'urn:mace:incommon:internet2.edu'

            ],
        ],
    ],

thanks!

Liam

[Dick Visser]

Hii

On Tue, 2 Jul 2024 at 20:57, Liam Hoekenga <li...@umich.edu> wrote:

and then it asks the MDQ server for my SSP's metadata (which seems like weird behavior to me)...
Jul 1 15:05:38 simplesamlphp INFO [4791e38f97] SimpleSAML\Metadata\Sources\MDQ: loading metadata entity [https://testsp.example.edu/ssp/] from [saml20-sp-remote]
Jul 1 15:05:38 simplesamlphp DEBUG [4791e38f97] SimpleSAML\Metadata\Sources\MDQ: downloading metadata for "https://testsp.example.edu/ssp/" from [https://mdq.incommon.org/entities/https%3A%2F%2Ftestsp.example.edu%2Fssp%2F]
Jul 1 15:05:39 simplesamlphp INFO [4791e38f97] Unable to fetch metadata for "https://testsp.example.edu/ssp/" from https://mdq.incommon.org/entities/https%3A%2F%2Ftestsp.example.edu%2Fssp%2F: file_get_contents(https://mdq.incommon.org/entities/https%3A%2F%2Ftestsp.example.edu%2Fssp%2F): Failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found

I *think* that you need to also have the standard metadata entry with 'flatfile' type, so that SSP can load its own metadata. I.e.:

    'metadata.sources' => [

        [

            'type' => 'flatfile',

             'directory => '/opt/local/simplesamlphp/metadata'

        ],

        [
            'type' => 'mdq',
            'server' => 'https://mdq.incommon.org',
            'validateCertificate' => [
                '/opt/local/simplesamlphp/cert/inc-md-cert-mdq.pem'
            ],
            'cachedir' => '/opt/local/var/cache/simplesamlphp/mdq',
            'cachelength' => 86400
        ],

    ],

It's not clear to me where discopower gets the list of IDPs to populate the discovery service.  It looks like the v1.xx disco.php used $config['idpdisco.external']['layout']['links'] ?

I did some tests myself but couldn't get things to work either... but that probably says more about me than the actual mdq/disco feature.

[Dick Visser]

On Tue, 2 Jul 2024 at 21:05, Dick Visser <dnmv...@gmail.com> wrote:

I did some tests myself but couldn't get things to work either... but that probably says more about me than the actual mdq/disco feature.

I was able to reproduce what you have. Hardwire to a specific IdP entityID and that gets successfully fetched with mdq.

But, I think that is about what the mdq protocol can do. Or at least, according to https://spaces.at.internet2.edu/display/MDQ/production-metadata, what https://mdq.incommon.org can do:

 > The service supports the MDQ protocol, which enables you to look up individual entity's metadata using its entity ID.

For the discovery part you'd still need to have the entire metadata, or at least a list of entityIDs - I think.

Hopefully someone else here can tell.

Dick

[Liam Hoekenga]

 > The service supports the MDQ protocol, which enables you to look up individual entity's metadata using its entity ID.

For the discovery part you'd still need to have the entire metadata, or at least a list of entityIDs - I think.

Adding 'flatfile' to the list of metadata sources allows the discovery interface to be displayed... but I think it's expecting a fully fleshed out metadata entry in saml20-idp-remote.php 

Caused by: Exception: saml20-idp-remote/'https://idp.educause.edu/idp/shibboleth'['SingleSignOnService']:Could not find a supported SingleSignOnService endpoint.

...instead of trying to fetch the metadata from MDQ.

Is it possible to define a metadata entry in saml20-idp-remote.php that uses MDQ?   ChatGPT and Gemini have suggested things that look like..

$metadata['urn:mace:incommon:internet2.edu'] = array(

    'mdq' => true,
    'mdq.server' => 'https://mdq.incommon.org/',
    'mdq.cache.duration' => 86400, // Cache duration in seconds
);

$metadata['https://idp.educause.edu/idp/shibboleth'] = array(
    'mdq' => 'https://mdq.incommon.org/',
    'sign.authnrequest' => true,
);

but neither of these ar right.

Liam

[Peter Schober]

Liam Hoekenga <li...@umich.edu> [2024-07-02 23:40 CEST]:

> Adding 'flatfile' to the list of metadata sources allows the discovery
> interface to be displayed... but I think it's expecting a fully fleshed out
> metadata entry in saml20-idp-remote.php

[...]

> ...instead of trying to fetch the metadata from MDQ.

Your's is quite a special case, I think, and I don't think you can
expect existing code to deal with these "new" ways of (not!) having
metadata around automagically. I.e., I don't know of any SAMLDS
implementation (including the Shibboleth EDS, SeamlessAccess or
SWITCHwayf) that can work based on solely on a list of selected
entityIDs to be fetched from some MDQ service endpoint.

With your SP set to use MDQ there's no metadata around for the SAMLDS
to process, as you've described. So unless the existing SAMLDS code is
changed to enable some other modes of operation (such as using a list
of entityIDs to fetch and cache metadata from MDQ, as you suggested)
it seems you're stuck with providing the metadata for (at least) the
desired IDPs to the SAMLDS code -- either by loading a full metadata
feed (making use of MDQ in only parts of the operation pointless) or
by finding a way to provide SimpleSAMLphp "metadata" (PHP arrays) for
the desired IDPs yourself, somehow.

One way to do the latter, I suppose, would be to regularly run
metarefresh yourself from a script, providing it with the MDQ URLs to
the IDPs you're interested in. That should give you SSP-style PHP
metadata for the desired IDPs and you should be able to feed those to
a SSP-based SAMLDS.
(Maybe you'd also have to merge either the XML source files or the
resulting PHP files into one "aggregate" for this to work with
existing code. The same goes for the method described below.)

Assuming there's a SAMLDS using the old DiscoJuice-style JSON metadata
format there's another way to do something similar: The pyff
distribution contains an XSLT file to create "discojson" from SAML 2.0 Metadata:
https://github.com/IdentityPython/pyFF/blob/master/src/pyff/xslt/discojson.xsl
(even though that process was later re-implemented in Python, as is
the rest of pyff).
Again you'd pull SAML 2.0 Metadata for the selected IDPs from the MDQ
server but then you'd use e.g. xsltproc (or any other XSLT
implementation) with the discojson.xsl above to produce
"discojson"-format JSON for the SAMLDS to consume.

HTH,
-peter

[Liam Hoekenga]

I got some additional advice from Patrick Radtke, and he suggested to go with the more traditional configuration (using the metarefresh and cron modules).  I've got that working now.  

I had hope that I could do something MDQ as to avoid having to process the InCommon IDP aggregate (which is pretty big), but alas.

thanks for your help!

Liam

[Pete Birkinshaw]

I might have something close to this -

I've got a simple discovery service in development that will take information in a small config file and doesn't need to consume metadata directly (with the huge caveat that doing this means it needs to be updated manually, and you need to know the details of a limited number of IdPs in advance) and will also soon have the option to use disco JSON or a new custom format too.

It's about two weeks old and not yet released as open source but if anyone wants to try it please let know and I'll get you access via Github or send the files - it's very easy to install. Hopefully it will be open-sourced in the next month or so.

I added a discofeed export to my metadata library Smee yesterday, so that will soon be another option for generating the data:

https://github.com/Digital-Identity-Labs/smee/commit/5a39734c35e657a46fdbd29e8e8e3a2c71b4c930

Pete

-- 
Pete Birkinshaw
Digital Identity Ltd | http://www.digitalidentity.ltd.uk 
Registered in England and Wales No. 7121888 

[Kevin Sandy]

If you’re okay with manually adding the IdPs, you can still avoid processing the entire InCommon feed. You can get the metadata for a single entity from https://mdq.incommon.org/entities/<url-encoded-entity-id>.

-- kevin

--
This is a mailing list for users of SimpleSAMLphp, not a support service. If you are willing to buy commercial support, please take a look here:
 
https://simplesamlphp.org/support
 
Before sending your question, make sure it is related to SimpleSAMLphp, and not your web server's configuration or any other third-party software. This mailing list cannot help with software that uses SimpleSAMLphp, only regarding SimpleSAMLphp itself.
 
Make sure to read the documentation:
 
https://simplesamlphp.org/docs/stable/
 
If you have an issue with SimpleSAMLphp that you cannot resolve and reading the documentation doesn't help, you are more than welcome to ask here for help. Subscribe to the list and send an email with your question. However, you will be expected to comply with some minimum, common sense standards in your questions. Please read this carefully:
 
http://catb.org/~esr/faqs/smart-questions.html
---
You received this message because you are subscribed to the Google Groups "SimpleSAMLphp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to simplesamlph...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/simplesamlphp/977bf41d-7552-401c-b24f-b51fdc8031edn%40googlegroups.com.