[Shib-Users] Unable to get Shibboleth SP working behind an SSL reverse-proxy
Mark Cave-Ayland
I've spent several days trying to get our Shibboleth SP working behind a
reverse proxy according to
https://spaces.internet2.edu/display/SHIB/SPReverseProxy without much
luck, and was hoping that someone here would be able to help me out.
After setting up the configuration as described above, it seems no
matter what I do, the SHIRE redirect to /Shibboleth.sso/SAML/POST is
always generated as a http rather than a https URL.
Here is the (simplified) Apache 2.2 configuration on the reverse proxy
itself:
<VirtualHost *:80>
ServerName mainsite.example.org
ProxyPass / http://backend.example.org/
ProxyPassReverse / http://backend.example.org/
</VirtualHost>
<VirtualHost _default_:443>
SSLEngine on
...etc...
<Location /Shibboleth.sso>
Order deny,allow
Allow from all
</Location>
<Location /shibboleth-sp>
Order deny,allow
Allow from all
</Location>
ProxyPass / http://backend.example.org:443/
ProxyPassReverse / http://backend.example.org:443/
</VirtualHost>
And here is the (simplified) Apache 2.2 configuration on the backend:
<VirtualHost *:80>
UseCanonicalName On
ServerName mainsite.example.org
<LocationMatch /protected_content>
AuthType shibboleth
ShibRequireSession On
require valid-user
</LocationMatch>
</VirtualHost>
<VirtualHost *:443>
UseCanonicalName On
ServerName https://mainsite.example.org:443
<Location /shibboleth-sp>
Allow from all
</Location>
</VirtualHost>
Also I've made the following changes to shibboleth2.xml on the backend:
ApplicationDefaults -> Sessions
handlerSSL="true"
My aim for the moment is just to have the /Shibboleth.sso URL running
under https with the remainder of the site under http, and then start
migrating other site content to https where deemed necessary.
Firstly, I've downloaded the metadata from
https://mainsite.example.org/Shibboleth.sso/Metadata and confirmed that
the AssertionConsumerServices are now represented by https rather than
http URLs - so far so good.
What happens at the moment when accessing
http://mainsite.example.org/protected_content is that I am directed to
my IDP where I can log in without any problem. However when generating
the SHIRE redirect URL, the generated URL is
http://mainsite.example.org/Shibboleth.sso/SAML/POST using http rather
than https (which of course fails because according to shibboleth2.xml
everything is set to listen on https).
Given that according to the Metadata the AssertionConsumerService URLs
are all https, I can't figure out why they are still generated
containing http. Does anyone have any more ideas as to why this is
happening?
Many thanks,
Mark.
--
Mark Cave-Ayland - Senior Technical Architect
PostgreSQL - PostGIS
Sirius Corporation plc - control through freedom
http://www.siriusit.co.uk
t: +44 870 608 0063
Sirius Labs: http://www.siriusit.co.uk/labs
Peter Schober
> I've spent several days trying to get our Shibboleth SP working behind a
> reverse proxy according to
> https://spaces.internet2.edu/display/SHIB/SPReverseProxy without much
> luck, and was hoping that someone here would be able to help me out.
> Here is the (simplified) Apache 2.2 configuration on the reverse proxy
> itself:
>
> <VirtualHost *:80>
> ServerName mainsite.example.org
>
> ProxyPass / http://backend.example.org/
> ProxyPassReverse / http://backend.example.org/
> </VirtualHost>
Doesn't this defeat the whole purpose when your proxy itself serves up
stuff on plain HTTP? I'd either dump this completely or at least
rewrite everything to HTTPS.
> <VirtualHost _default_:443>
>
> SSLEngine on
> ...etc...
>
> <Location /Shibboleth.sso>
> Order deny,allow
> Allow from all
> </Location>
> <Location /shibboleth-sp>
> Order deny,allow
> Allow from all
> </Location>
>
> ProxyPass / http://backend.example.org:443/
> ProxyPassReverse / http://backend.example.org:443/
> </VirtualHost>
When you're offloading SSL (as the subject of your email implies) why
proxy to HTTPS on the backend? And http://...:443/ is s strange thing
in itself.
> And here is the (simplified) Apache 2.2 configuration on the backend:
>
> <VirtualHost *:80>
>
> UseCanonicalName On
> ServerName mainsite.example.org
>
> <LocationMatch /protected_content>
> AuthType shibboleth
> ShibRequireSession On
> require valid-user
> </LocationMatch>
> </VirtualHost>
>
> <VirtualHost *:443>
>
> UseCanonicalName On
> ServerName https://mainsite.example.org:443
>
> <Location /shibboleth-sp>
> Allow from all
> </Location>
> </VirtualHost>
I'd throw away the vhost on 443 (since I'd terminate SSL on the
reverse proxy and then proxy plain HTTP to the backend). And your port
80 vhost is missing the relevant bit that is clearly documented on the
very page you cite above (virtualizing scheme and port in the port 80
vhost).
Note that none of this has anything to do with Shibboleth.
-peter
Scott Cantor
> Given that according to the Metadata the AssertionConsumerService URLs
> are all https, I can't figure out why they are still generated
> containing http. Does anyone have any more ideas as to why this is
> happening?
If the metadata says https and the message to the IdP says to use http, then
it would stop there with an error unless it had no metadata at all about the
SP was treating it as unrecognized. So I'm not following you.
Using handlerSSL="true" guarantees that any ACS URL generated in the request
will be https, no matter what the server tells the SP the physical state is.
That's one of the few cases where a misconfigured vhost won't even matter in
terms of the generated URL.
So my guess is that it is https in both the metadata and in the request, and
therefore starts as https on the final hop, and *then* it's being routed for
some reason to the vhost on the back end such that the SP thinks the request
is http and not https.
What do the logs say is being mapped for each request?
-- Scott
Peter Schober
> are all https, I can't figure out why they are still generated
> containing http. Does anyone have any more ideas as to why this is
> happening?
Because the port 80-vhost on the backend hasn't been told otherwise.
Since the Shib SP, which generates the AuthnRequest on the backend, is
relying on Apache httpd for self-referencing URLs the request contains
(non-virtualized) http-URLs for the ACS.
-peter
Scott Cantor
>> Given that according to the Metadata the AssertionConsumerService URLs
>> are all https, I can't figure out why they are still generated
>> containing http. Does anyone have any more ideas as to why this is
>> happening?
>
> Because the port 80-vhost on the backend hasn't been told otherwise.
>
> Since the Shib SP, which generates the AuthnRequest on the backend, is
> relying on Apache httpd for self-referencing URLs the request contains
> (non-virtualized) http-URLs for the ACS.
That's true if handlerSSL isn't on, but assuming it is, they *should* be
https. I think so, anyway. I think even the metadata generator will honor
that when it asks the system to derive the right base location for the
handlers to use.
I certainly agree with the suggestions to just route it all to http on the
backend and simplify the setup, though I was still trying to see what's
going wrong.
-- Scott
Mark Cave-Ayland
> Doesn't this defeat the whole purpose when your proxy itself serves up
> stuff on plain HTTP? I'd either dump this completely or at least
> rewrite everything to HTTPS.
Oh sure. For the moment this is a work in progress, and so my aim is to
start migrating specific application URLs as soon as I have the proof of
concept working.
>> <VirtualHost _default_:443>
>>
>> SSLEngine on
>> ...etc...
>>
>> <Location /Shibboleth.sso>
>> Order deny,allow
>> Allow from all
>> </Location>
>> <Location /shibboleth-sp>
>> Order deny,allow
>> Allow from all
>> </Location>
>>
>> ProxyPass / http://backend.example.org:443/
>> ProxyPassReverse / http://backend.example.org:443/
>> </VirtualHost>
> When you're offloading SSL (as the subject of your email implies) why
> proxy to HTTPS on the backend? And http://...:443/ is s strange thing
> in itself.
Yeah. The SSL offload is happening on the reverse proxy, and as I
understand it, should forward the request over plain http to port 443 on
the backend.
> I'd throw away the vhost on 443 (since I'd terminate SSL on the
> reverse proxy and then proxy plain HTTP to the backend). And your port
> 80 vhost is missing the relevant bit that is clearly documented on the
> very page you cite above (virtualizing scheme and port in the port 80
> vhost).
Oh that's interesting. The reason I had it set up like was this was
because I found that with handlerURL set to "https:///Shibboleth.sso"
then without using a virtual host on port 443, the port number would
appear incorrect in the generated Metadata. I can't work out at the
moment what the difference is between prefixing the handlerURL with
https:// compared with setting handlerSSL to false.
If I can get things to work without the second vhost then I've got
absolutely not objection to that.
> Note that none of this has anything to do with Shibboleth.
That could be entirely likely ;) I suspect I'm misunderstanding
something simple with respect to the configuration.
ATB,
Peter Schober
>> When you're offloading SSL (as the subject of your email implies) why
>> proxy to HTTPS on the backend? And http://...:443/ is s strange thing
>> in itself.
>
> Yeah. The SSL offload is happening on the reverse proxy, and as I
> understand it, should forward the request over plain http to port 443 on
> the backend.
A more sane way to do things (given that we have perfectly well
established IANA assignments changing ports around will only serve to
confuse you and/or your successors and co-workers) would be to forward
requests over plain http to port 80 on the backend. And virtualize
schema and port there as per the httpd docs (and said wiki page).
> Oh that's interesting. The reason I had it set up like was this was
> because I found that with handlerURL set to "https:///Shibboleth.sso"
> then without using a virtual host on port 443, the port number would
> appear incorrect in the generated Metadata. I can't work out at the
> moment what the difference is between prefixing the handlerURL with
> https:// compared with setting handlerSSL to false.
I would hope that implementing above (and prior) suggestings you'd be
able to start with a correctly virtualized httpd config, as well as a
clean (i.e., shibboleth2.xml.dist) Shibboleth SP config and will find
that things will just work without messing with handlerSSL at all.
-peter
Mark Cave-Ayland
> If the metadata says https and the message to the IdP says to use http, then
> it would stop there with an error unless it had no metadata at all about the
> SP was treating it as unrecognized. So I'm not following you.
Yeah, I've seen references to this while researching different pages on
the wiki but figured I was running into something different.
> Using handlerSSL="true" guarantees that any ACS URL generated in the request
> will be https, no matter what the server tells the SP the physical state is.
> That's one of the few cases where a misconfigured vhost won't even matter in
> terms of the generated URL.
Okay, that's interesting to know.
> So my guess is that it is https in both the metadata and in the request, and
> therefore starts as https on the final hop, and *then* it's being routed for
> some reason to the vhost on the back end such that the SP thinks the request
> is http and not https.
>
> What do the logs say is being mapped for each request?
Hmmmm. AFAICT the http request is being generated directly, or at least
I can't find a preceding https request looking through the log files.
ATB,
Scott Cantor
> I can't find a preceding https request looking through the log files.
You're not using the SP logs or following what I'm asking about, but my
suggestion is to listen to Peter and just start over with the proper proxy
configuration and it won't matter.
-- Scott
Mark Cave-Ayland
> A more sane way to do things (given that we have perfectly well
> established IANA assignments changing ports around will only serve to
> confuse you and/or your successors and co-workers) would be to forward
> requests over plain http to port 80 on the backend. And virtualize
> schema and port there as per the httpd docs (and said wiki page).
Unfortunately I don't think I can do this because the backend server
containing all of the insecure content is the same as the backend server
hosting the Shibboleth instance. This is why I have a second virtual
host running on port 443, since I can't set ServerName to
https://mainsite.example.org:443 for the virtual host running on port 80
without breaking the insecure site.
> I would hope that implementing above (and prior) suggestings you'd be
> able to start with a correctly virtualized httpd config, as well as a
> clean (i.e., shibboleth2.xml.dist) Shibboleth SP config and will find
> that things will just work without messing with handlerSSL at all.
At the moment with the second virtual host running on port 443, setting
the ServerName to https://mainsite.example.org:443 and prefixing
handlerURL with https:// automatically generates the correct https URLs
for the metadata, plus allows me to browse to https URLs under
/Shibboleth.sso/ without reporting any Location/Binding errors. But
*still* the /SAML2/POST URL is generated with a plain http...
Scott Cantor
> the ServerName to https://mainsite.example.org:443 and prefixing
> handlerURL with https:// automatically generates the correct https URLs
> for the metadata, plus allows me to browse to https URLs under
> /Shibboleth.sso/ without reporting any Location/Binding errors. But
> *still* the /SAML2/POST URL is generated with a plain http...
Generated by what? What do your AuthnRequests actually look like?
Also, don't prefix handlerURL. If it's not /Shibboleth.sso, change it back.
That is not likely to be a fix for whatever your issue is.
-- Scott
Mark Cave-Ayland
>> At the moment with the second virtual host running on port 443, setting
>> the ServerName to https://mainsite.example.org:443 and prefixing
>> handlerURL with https:// automatically generates the correct https URLs
>> for the metadata, plus allows me to browse to https URLs under
>> /Shibboleth.sso/ without reporting any Location/Binding errors. But
>> *still* the /SAML2/POST URL is generated with a plain http...
>
> Generated by what? What do your AuthnRequests actually look like?
Well, when I attempt to access a protected URL, I get redirected to my
IDP where I can log in successfully using my username and password. Once
logged in, I get redirected to a URL of the form
https://identity.example.org/idp/Authn/UserPassword, followed by a
redirection to http://mainsite.example.org/Shibboleth.sso/SAML2/POST.
Looking at the information on
https://spaces.internet2.edu/display/SHIB/SPReverseProxy, I'd guess that
I am failing on step 6.
> Also, don't prefix handlerURL. If it's not /Shibboleth.sso, change it back.
> That is not likely to be a fix for whatever your issue is.
Okay - I've removed the prefix, restarted both shibd and Apache, and it
looks like I can still browse to
https://mainsite.example.org/Shibboleth.sso/SAML2/POST without getting
any Location errors, plus the Metadata also contains https prefix URLs :)
Scott Cantor
>
> Well, when I attempt to access a protected URL, I get redirected to my
> IDP where I can log in successfully using my username and password. Once
> logged in, I get redirected to a URL of the form
> https://identity.example.org/idp/Authn/UserPassword, followed by a
> redirection to http://mainsite.example.org/Shibboleth.sso/SAML2/POST.
That isn't my question. You need to see what's in the XML message the IdP
receives in its log by upping the trace level, or switch to SAML 1 so that
the message is a plain redirect and easier to diagnose.
-- Scott
Mark Cave-Ayland
> That isn't my question. You need to see what's in the XML message the IdP
> receives in its log by upping the trace level, or switch to SAML 1 so that
> the message is a plain redirect and easier to diagnose.
Putting the IDP in TRACE mode gives the following for the AuthnRequest
from the SP to the IDP:
<?xml version="1.0" encoding="UTF-8"?><samlp:AuthnRequest
xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol"
AssertionConsumerServiceIndex="1"
Destination="https://identity.example.org/idp/profile/SAML2/Redirect/SSO"
ID="_42a6d4015afc56b65cd6bbc9bd48c8c5"
IssueInstant="2010-02-19T00:15:43Z" Version="2.0"><saml:Issuer
xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion">https://mainsite.example.org/shibboleth</saml:Issuer><samlp:NameIDPolicy
AllowCreate="1"/></samlp:AuthnRequest>
Looking through the IDP log I also see the following interesting
snippets (note the http rather than https URLs):
00:17:13.408 - DEBUG
[org.opensaml.saml2.binding.AuthnResponseEndpointSelector:100] -
Filtering peer endpoints. Supported peer endpoint bindings:
[urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST-SimpleSign,
urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST,
urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Artifact]
00:17:13.408 - DEBUG
[org.opensaml.saml2.binding.AuthnResponseEndpointSelector:115] -
Removing endpoint http://mainsite.example.org/Shibboleth.sso/SAML2/ECP
because its binding urn:oasis:names:tc:SAML:2.0:bindings:PAOS is not
supported
...
...
00:17:13.909 - DEBUG
[org.opensaml.saml2.binding.encoding.HTTPPostEncoder:121] - Invoking
Velocity template to create POST body
00:17:13.910 - DEBUG
[org.opensaml.saml2.binding.encoding.HTTPPostEncoder:152] - Encoding
action url of: http://mainsite.example.org/Shibboleth.sso/SAML2/POST
Scott Cantor
> Putting the IDP in TRACE mode gives the following for the
> AuthnRequest from the SP to the IDP:
Your request does NOT say to use https, it says to use what's in the metadata.
> Looking through the IDP log I also see the following interesting
> snippets (note the http rather than https URLs):
Your vhost configuration, though complicated, looked correct to me, and I suspected that one or both of the claims you made up front were wrong. The request doesn't say what you thought it did, and the metadata isn't correct, which combined to create the error.
-- Scott
Mark Cave-Ayland
> > Putting the IDP in TRACE mode gives the following for the
> > AuthnRequest from the SP to the IDP:
>
> You're sending the ACS location by index, as I suspected, so the SP is
> either a bit out of date or acsByIndex is set to true, or both. Use
> false, it creates annoying debugging problems like this.
>
> Your request does NOT say to use https, it says to use what's in the
> metadata.
Arrrgh - I didn't realise that the IDP kept it's own version of the
metadata! Not being particularly involved in Shibboleth before, I guess
I thought that the IDP would download (and cache) the SP metadata during
it's first communication. So I've been looking in the wrong place all
along...
> Yes, because the metadata is wrong, as I also suspected.
>
> Your vhost configuration, though complicated, looked correct to me, and
> I suspected that one or both of the claims you made up front were wrong.
> The request doesn't say what you thought it did, and the metadata isn't
> correct, which combined to create the error.
Yes. I've now got my original vhost configuration working brilliantly,
with the SP reporting the correct secure URLs, and the entire login
process running correctly over https. Thanks a lot to both you and Peter
for your help and patience given that I am totally new to all this - it
really is very much appreciated.
ATB,
M-D.
Scott Cantor
> Arrrgh - I didn't realise that the IDP kept it's own version of the
> metadata! Not being particularly involved in Shibboleth before, I guess
> I thought that the IDP would download (and cache) the SP metadata during
> it's first communication. So I've been looking in the wrong place all
> along...
I don't know at what stage you're in deployment-wise, but if that idea
sounded reasonable, you may need to review this material:
https://spaces.internet2.edu/display/SHIB2/TrustManagement
-- Scott