[Shib-Users] SP and Apache reverse proxy
Andy Swiffin
I realise this has been raised before, but I don't see any answers to the questions I have. Also, what I have working doesn't match up with what is documented at https://spaces.internet2.edu/display/SHIB2/SPReverseProxy and I thought I would ask here for some insight.
In order to investigate how to provide a shibboleth SP service to an Apache reverse Proxy I have a mockup where:
browser ->spproxy.dundee.ac.uk ->sptarget.dundee.ac.uk
I have set up sptarget.dundee.ac.uk to have:
ServerName spproxy.dundee.ac.uk:80
UseCanonicalName On
in its httpd.conf and also the same name for 443
I have also put spproxy.dundee.ac.uk in all of the relevant places in shibboleth2.xml (except the entityID) and initially I have put
handlerURL="/Shibboleth.sso" handlerSSL="false"
in the <Sessions> element. To match with that I have put http://spproxy.dundee.ac.uk/Shibboleth.sso/etcetc in the appropriate ACSs in the metadata. Finally in the reverse proxy server spproxy I have put the the following in it's httpd.conf:
ProxyPass /Shibboleth.sso/ http://sptarget.dundee.ac.uk/Shibboleth.sso/
ProxyPassReverse /Shibboleth.sso/ http://sptarget.dundee.ac.uk/Shibboleth.sso/
ProxyPass /foo/ http://sptarget.dundee.ac.uk/foo/
ProxyPassReverse /foo/ http://sptarget.dundee.ac.uk/foo/
------
This all works fine!
BUT, I noted that it was at variance with the documentation I quoted at the start which says:
in Applications -> Sessions set (or leave the default) handlerURL="/Shibboleth.sso" and set handlerSSL="true"
(and also has metadata with https:// to match that)
If I do this, things don't work. After accessing spproxy.dundee.ac.uk/foo I get redirected to the login page, authenticate and then get left at a 404 page for: https://spproxy.dundee.ac.uk/Shibboleth.sso/SAML/POST (or /SAML2/ if I set it to prefer that)
Checking the Apache logs for sptarget shows that this has been passed through to there OK:
134.36.40.227 - - [04/Aug/2010:15:12:04 +0100] "POST /Shibboleth.sso/SAML/POST HTTP/1.1" 404 222
and
[Wed Aug 04 15:12:04 2010] [error] [client 134.36.40.227] File does not exist: C:/apache2.2/htdocs/Shibboleth.sso, referer: https://idp.dundee.ac.uk/idp/Authn/UserPassword
Now this does seem logical to me, I've told it handlerSSL=true so I would expect that it would be expecting https?
Despite the documentation saying the opposite I decided to try:
ProxyPass /Shibboleth.sso/ https://sptarget.dundee.ac.uk/Shibboleth.sso/
ProxyPassReverse /Shibboleth.sso/ https://sptarget.dundee.ac.uk/Shibboleth.sso/
and provided that I remember to set SSLProxyEngine on everything now works again.
SO, my question is simply: Have I done something daft that stops it working as documented or is the documentation not quite correct?
-------
I now have another unrelated question:
I have no problem whatsoever reverse proxying
sproxy.dundee.ac.uk/foo -> sptarget.dundee.ac.uk/foo
where the same endpoint foo is used on the proxy and on the target, but what if I want to do:
ProxyPass /foo/ http://sptarget.dundee.ac.uk/bar/
ProxyPassReverse /foo/ http://sptarget.dundee.ac.uk/bar/
Is there any way of doing this? On return from authentication it wants to go to sproxy.dundee.ac.uk/bar/ which is unknown and so a 404 is thrown up. One way of getting around this might be to then add
ProxyPass /bar/ http://sptarget.dundee.ac.uk/bar/
ProxyPassReverse /bar/ http://sptarget.dundee.ac.uk/bar/
but then the user at the browser sees spproxy.dundee.ac.uk/bar/index.html when they are returned after authentication despite the fact that they originally typed spproxy.dundee.ac.uk/foo/index.html? I don't see how there is a way of fixing this because the IdP must return them to where it thinks they started from, which is /bar/ not /foo/ ?
TIA
Andy Swiffin
Dundee
Scotland
************************************************************
Please consider the environment. Do you really need to print this email?
The University of Dundee is a registered Scottish charity, No: SC015096
Peter Schober
(instead of composing a new email for a new request) but since this
touches on stuff I've written...
* Andy Swiffin <a.l.s...@dundee.ac.uk> [2010-08-04 16:54]:
> I have set up sptarget.dundee.ac.uk to have:
>
> ServerName spproxy.dundee.ac.uk:80
> UseCanonicalName On
>
> in its httpd.conf and also the same name for 443
I only have httpd listening on port 80 on the resource/target and
hence only have a single vhost on port 80, virtualizing both schema
and port exactly as specified on the wiki page. But YMMV.
> This all works fine!
Glad to hear that.
> BUT, I noted that it was at variance with the documentation I quoted at the start which says:
>
> in Applications -> Sessions set (or leave the default) handlerURL="/Shibboleth.sso" and set handlerSSL="true"
>
> (and also has metadata with https:// to match that)
This has been in there since Sep 11, 2007 (in the Shib1 space, from
which the content was copied over to the current documentation) and if
you're doing SSL offloading on the proxy handlerSSL="true" also seems
wrong to me -- were it not like that in my working config. So I guess
this probably is an effect of our locally hacked mod_shib.
Feel free to correct this in the wiki (which is what I suggested the
last time you brought this up months ago, IIRC).
The metadata of course will always have https endpoints (unless the SP
is completely insecure), since that is where the HTTP user agent will
connect to the proxy (e.g. with HTTP POST).
> Despite the documentation saying the opposite I decided to try:
>
> ProxyPass /Shibboleth.sso/ https://sptarget.dundee.ac.uk/Shibboleth.sso/
> ProxyPassReverse /Shibboleth.sso/ https://sptarget.dundee.ac.uk/Shibboleth.sso/
>
> and provided that I remember to set SSLProxyEngine on everything now works again.
Sure, if you go from https/proxy to https/resource this will also work
with handlerSSL="true", since then requests will actually be on https
even on the proteced resource.
cheers,
-peter
Peter Schober
> I now have another unrelated question:
> but what if I want to do:
>
> ProxyPass /foo/ http://sptarget.dundee.ac.uk/bar/
> ProxyPassReverse /foo/ http://sptarget.dundee.ac.uk/bar/
>
> Is there any way of doing this? On return from authentication it
> wants to go to sproxy.dundee.ac.uk/bar/ which is unknown and so a
> 404 is thrown up.
See relayState on
https://spaces.internet2.edu/display/SHIB2/NativeSPSessionInitiator#NativeSPSessionInitiator-CommonAttributes
If the relaystate in stored a cookie (default) you might be able to
rewrite that cookie's value on the proxy (with mod_rewrite), but note
that the cookie name itself is dynamic (called
_shibstate_<something>), so a regex migh be in order.
Same when you leave out the relayState parameter, then it wiill be
passed as a request parameter on the SSO URL after the SAMLRequest,
something like
"GET /idp/profile/SAML2/Redirect/SSO?SAMLRequest=<...>&RelayState=https%3A%2F%2Fspproxy.dundee.ac.uk%2Fbar%2F"
So, again, the proxy might be able to rewrite this URL via mod_rewrite.
If the relayState on that SP is put into the SPs storage service
you're probably SOL.
cheers,
-peter
Scott Cantor
working
> as documented or is the documentation not quite correct?
I believe you've left off the scheme virtualization that tells the back end
that even though it's http, the effective scheme is https.
> where the same endpoint foo is used on the proxy and on the target, but
what
> if I want to do:
>
> ProxyPass /foo/ http://sptarget.dundee.ac.uk/bar/
> ProxyPassReverse /foo/ http://sptarget.dundee.ac.uk/bar/
>
> Is there any way of doing this? On return from authentication it wants
to
> go to sproxy.dundee.ac.uk/bar/ which is unknown and so a 404 is thrown up.
As Peter said, you want a reverse proxy, you need to be one. That means
rewriting URLs, cookies, content, etc. Personally, I think it's a serious
pain.
-- Scott
Andy Swiffin
<2010080416...@wssp.cc.univie.ac.at>, Peter Schober
<peter....@univie.ac.at> wrote:
> I usually don't reply anymore to people hi-jacking any old thread
> (instead of composing a new email for a new request) but since this
> touches on stuff I've written...
Could you explain how this is hi-jacking an old thread?
This is a totally newly written email, with a new subject line, on a topic I'm working currently working on. Are you usually so rude to people you meet in the street or do you reserve it for here?
> This has been in there since Sep 11, 2007 (in the Shib1 space, from
> which the content was copied over to the current documentation) and if
> you're doing SSL offloading on the proxy handlerSSL="true" also seems
> wrong to me -- were it not like that in my working config. So I guess
> this probably is an effect of our locally hacked mod_shib.
> Feel free to correct this in the wiki (which is what I suggested the
> last time you brought this up months ago, IIRC).
So I think we've established that as documented this isn't right? I was hoping to get some input from other people who had done this to see what they had done, just to confirm that I was on the right track in an area that is totally new to me. Yes, I wouldn't have bothered with handlerssl=true either, but those who were deemed to be wise who had gone there before me had done this so this is what I tried first. But thanks for the clarification that this may not be the case.
> The metadata of course will always have https endpoints (unless the SP
> is completely insecure), since that is where the HTTP user agent will
> connect to the proxy (e.g. with HTTP POST).
I find that if you have handlerSSL=false the metadata will have http:// endpoints.
Thankyou for your clarification, it's nice to know that I was doing the right thing. As its your documentation I'll let you correct it or add a caveat that it may be erroneous do to your "locally hacked mod_shib".
I'd still be interested to hear from other people who've been down this path as it seems one of the more convoluted deployments with pitfalls.
Regards
Andy Swiffin
--
*********
Andy Swiffin
Senior Network Specialist, Corporate Information systems
Information & Communications Services (ICS)
University of Dundee, Computing Centre, Park Place, Dundee, DD1 4HN
Direct: 01382 388000 (Service Desk)
Visit our website at: www.dundee.ac.uk/ics
*********
Andy Swiffin
"Scott Cantor" <cant...@osu.edu> wrote:
>> SO, my question is simply: Have I done something daft that stops it
> working
>> as documented or is the documentation not quite correct?
>
> I believe you've left off the scheme virtualization that tells the back end
> that even though it's http, the effective scheme is https.
>
> rewriting URLs, cookies, content, etc. Personally, I think it's a serious
> pain.
>
> -- Scott
Thanks for that.
Do you (or anyone else) have a sample configuration or cookbook that's posted anywhere, I now understand that what's documented may be incomplete.
I'm new to this area, effectively I just provide an identity service to my "customers" who run the web stuff, it isn't anything I've had to deal with before. However as per usual I find I have to research all the possibilities and understand how "their world" works in order to provide them with the best solutions.
Regards
Andy Swiffin
Peter Schober
> >>> On 04/08/2010 at 17:18, in message
> <2010080416...@wssp.cc.univie.ac.at>, Peter Schober
> <peter....@univie.ac.at> wrote:
> > I usually don't reply anymore to people hi-jacking any old thread
> > (instead of composing a new email for a new request) but since this
> > touches on stuff I've written...
>
> Could you explain how this is hi-jacking an old thread?
>
> This is a totally newly written email, with a new subject line, on a
> topic I'm working currently working on. Are you usually so rude to
> people you meet in the street or do you reserve it for here?
Actually that's the definition of thread hi-jacking, but read on:
You replied to an existing email from a completele different thread (a
seqeuence of emails all dealing with the same topic, unrelated to your
question). By that your mail user agent inserts an In-Reply-To:
RFC2822 header, which is then interpreted by other software. (Check
this emails headers now, you'll see).
Mail user agents may choose to present this message in some hierarchy
or relation to the other ("hi-jacked") thread, as will most archiving
software for the web. Some will not. Most are configurable.
This is a function of you hitting "reply" to an existing message.
The value of the subject header does not factor into it.
I only mentioned this because I've asked people several times (on this
list) to not hit "reply" when the message is in fact *not* a *reply*.
Which was not welcome at all, so I chose to just ignore questions from
those who insisted on reusing old threads for new requests.
So as far as netiquette is concerned this is rude behaviour. And I
don't see how pointing out a specific behaviour (replying when it's
not a reply) is being rude to "people I meet in the street".
Others have chosen to accept the general decline in the ability to
write usable emails (several pages of quoted text with a single line
added above; broken threading by hitting reply; not trimming the reply
to the relevant parts, etc.). So I don't lecture anymore (at least not
on this list ;) But I chose to ignore such posts completly. And no, I
don't think it will make any difference. But I still get to decide how
to spend my time.
> So I think we've established that as documented this isn't right? I
> was hoping to get some input from other people who had done this to
> see what they had done, just to confirm that I was on the right
> track in an area that is totally new to me.
By definition ("other people") I can't help you here, sorry.
> I find that if you have handlerSSL=false the metadata will have
> http:// endpoints.
The metadata contains what you want it to contain. Please let go of
the idea that the output of the metadata handler may or must not be
modified. It's a starting point. And SSL offloading is not a use case
it was particularly designed for.
cheers,
-peter
Peter Schober
> Do you (or anyone else) have a sample configuration or cookbook
> that's posted anywhere, I now understand that what's documented may
> be incomplete.
>
> I'm new to this area, effectively I just provide an identity service
> to my "customers" who run the web stuff, it isn't anything I've had
> to deal with before. However as per usual I find I have to
> research all the possibilities and understand how "their world"
> works in order to provide them with the best solutions.
This really has nothing to do with Shibboleth -- the question now has
become: How do I modify the value of an HTTP response header (in case
of relayState being set in a cookeie) after the answer has beed
provided by mod_proxy (but before returning the reply to the user
agent).
You're now knee-deep in Apache httpd internals. From a quick search
I'd say using mod_headers and mod_proxy in combination does not look
too promising (but don't take my word for anything), so I'd probably
go for mod_perl and something like
http://perl.apache.org/docs/2.0/api/Apache2/Filter.html
http://search.cpan.org/~pgollucci/Apache2-Filter-HTTPHeadersFixup-0.06/
Let us know how this works out, though.
-peter
Andy Swiffin
>> I find that if you have handlerSSL=false the metadata will have
>> http:// endpoints.
>
> The metadata contains what you want it to contain. Please let go of
> the idea that the output of the metadata handler may or must not be
> modified. It's a starting point. And SSL offloading is not a use case
> it was particularly designed for.
>
Let me rephrase that as I haven't even mentioned the output of the metadata handler, which I haven't used, so its irrelevant to this discussion:
I find that if you have handlerSSL=false the metadata MUST have http:// endpoints, otherwise it doesn't work, you'll get the familiar "no peer endpoint" message.
Just thought I'd clarify that in case anyone else goes here.
Regards
Andy Swiffin
Peter Schober
> I find that if you have handlerSSL=false the metadata MUST have
> http:// endpoints, otherwise it doesn't work, you'll get the
> familiar "no peer endpoint" message.
>
> Just thought I'd clarify that in case anyone else goes here.
I gave the document an overhaul, working in all of your comments
https://spaces.internet2.edu/display/SHIB2/SPReverseProxy
except for the one above, which I've seen only now and which I also
think to be not correct.
-peter
Scott Cantor
metadata
> handler, which I haven't used, so its irrelevant to this discussion:
>
> I find that if you have handlerSSL=false the metadata MUST have http://
> endpoints, otherwise it doesn't work, you'll get the familiar "no peer
> endpoint" message.
Changing the handlerSSL setting has nothing to do with the metadata unless
you're, yes, generating it with the handler.
All that setting does is force the SP to set the ACS endpoint it askes the
IdP to use to an SSL version even if the resource is http. Without it, the
scheme is set based on the request URL.
With false, and without offloading, the metadata would have to carry both
sets of endpoints. With true, or with offloading, only SSL would be needed.
That's it.
-- Scott
Andy Swiffin
<2010080513...@wssp.cc.univie.ac.at>, Peter Schober
<peter....@univie.ac.at> wrote:
> * Andy Swiffin <a.l.s...@dundee.ac.uk> [2010-08-05 12:57]:
>> I find that if you have handlerSSL=false the metadata MUST have
>> http:// endpoints, otherwise it doesn't work, you'll get the
>> familiar "no peer endpoint" message.
>>
>> Just thought I'd clarify that in case anyone else goes here.
>
> I gave the document an overhaul, working in all of your comments
> https://spaces.internet2.edu/display/SHIB2/SPReverseProxy
Very good.
You've made no mention of handlerSSL="false" which I think you will need if your proxypass (etc) for Shibboleth.sso is to http://
> except for the one above, which I've seen only now and which I also
> think to be not correct.
yes, you're right its only partially correct now I think about it, as Scott says:
> All that setting does is force the SP to set the ACS endpoint it askes the
> IdP to use to an SSL version even if the resource is http. Without it, the
> scheme is set based on the request URL.
> With false, and without offloading, the metadata would have to carry both
> sets of endpoints. With true, or with offloading, only SSL would be needed
But as you're only talking http to the backend just http:// would do in the metadata.
It seems to work OK for me that way anyway.
Thankyou for your input, I'm reassured that things will now work as I expect.
Peter Schober
> > I gave the document an overhaul, working in all of your comments
> > https://spaces.internet2.edu/display/SHIB2/SPReverseProxy
>
> Very good.
>
> You've made no mention of handlerSSL="false" which I think you will
> need if your proxypass (etc) for Shibboleth.sso is to http://
That was on purpose since I thought that was the default setting but I
see now that it's not. I'll add that back in.
> But as you're only talking http to the backend just http:// would do
> in the metadata.
No. The metadata (here) deals with how the IdP will get messages sent
to the proxy, since the proxy is all the HTTP User Agent will ever
interact with. Hence you shoud have HTTPS endpoints and *only* HTTPS
endpoints in the metadata, IMO. Unless, as now indicated on the wiki
page, you're running HTTP-only (in which case all of this doesn't
apply, as I said back when we had this conversation elsewhere).
-peter