Again, for the record. We had difficulties with authentication sessions timing out (we'd need to logout and login again to our Microsoft accounts). Also, the FederatedMetadata.xml would occasionally be updated (on the Microsoft end). As the plugin can not read the update from the URL, we would need to turn-off global security, update the SAML configuration and then reconfigure role assignments.
We instead switched to authenticating in apache with mod_auth_openidc and using the Jenkins Reverse Proxy Auth plugin.