Problem attempting to use Apache 2.2 mod_proxy_http and mod_shib w/ShibUseHeaders simultaneously

[Steve Siirila]

As the subject describes, we are having a problem when attempting to use mod_shib w/ShibUseHeaders in conjunction with mod_proxy_http (proxying to a non-apache web service listening on localhost).

The application running on localhost sees the following HTTP headers in response to a GET and a POST request when NOT using "ShibUseHeaders" ('###' and xxx.yyy used below to mask potentially private data):

Jan 25 11:35:46 [14013] HTTP GET request, URL: /session/###
Jan 25 11:35:46 [14013] Header: accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Jan 25 11:35:46 [14013] Header: accept-language: en-us,en;q=0.5
Jan 25 11:35:46 [14013] Header: x-forwarded-server: #####.umn.edu
Jan 25 11:35:46 [14013] Header: user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1
Jan 25 11:35:46 [14013] Header: connection: Keep-Alive
Jan 25 11:35:46 [14013] Header: referer: https://#####.umn.edu/session/###
Jan 25 11:35:46 [14013] Header: host: localhost
Jan 25 11:35:46 [14013] Header: accept-encoding: gzip, deflate
Jan 25 11:35:46 [14013] Header: x-forwarded-host: #####.umn.edu
Jan 25 11:35:46 [14013] Header: dnt: 1
Jan 25 11:35:46 [14013] Header: cookie: testuser=###; umnAuthV2=###; _shibsession_###=_###
Jan 25 11:35:46 [14013] Header: x-forwarded-for: 63.231.xxx.yyy
Jan 25 11:35:46 [14013] Header: accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

Jan 25 11:35:49 [14054] HTTP POST request, URL: /session/###
Jan 25 11:35:49 [14054] Header: accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Jan 25 11:35:49 [14054] Header: accept-language: en-us,en;q=0.5
Jan 25 11:35:49 [14054] Header: content-type: application/x-www-form-urlencoded
Jan 25 11:35:49 [14054] Header: x-forwarded-server: ###.umn.edu
Jan 25 11:35:49 [14054] Header: user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1
Jan 25 11:35:49 [14054] Header: connection: Keep-Alive
Jan 25 11:35:49 [14054] Header: referer: https://###.umn.edu/session/###
Jan 25 11:35:49 [14054] Header: content-length: 54
Jan 25 11:35:49 [14054] Header: host: localhost
Jan 25 11:35:49 [14054] Header: accept-encoding: gzip, deflate
Jan 25 11:35:49 [14054] Header: x-forwarded-host: ###.umn.edu
Jan 25 11:35:49 [14054] Header: cookie: testuser=###; umnAuthV2=###; _shibsession_###=_###
Jan 25 11:35:49 [14054] Header: x-forwarded-for: 63.231.xxx.yyy
Jan 25 11:35:49 [14054] Header: accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

However, adding the single configuration line "ShibUseHeaders on" causes a different sent of HTTP headers to be seen on the same GET and POST requests:

Jan 25 11:34:32 [13946] HTTP GET request, URL: /session/###
Jan 25 11:34:32 [13946] Header: accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Jan 25 11:34:32 [13946] Header: accept-language: en-us,en;q=0.5
Jan 25 11:34:32 [13946] Header: user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1
Jan 25 11:34:32 [13946] Header: referer: https://###.umn.edu/session/###
Jan 25 11:34:32 [13946] Header: shib-session-id: _###
Jan 25 11:34:32 [13946] Header: host: localhost
Jan 25 11:34:32 [13946] Header: shib-authncontext-class: urn:oasis:names:tc:SAML:2.0:ac:classes:unspecified
Jan 25 11:34:32 [13946] Header: shib-session-index: ###
Jan 25 11:34:32 [13946] Header: accept-encoding: gzip, deflate
Jan 25 11:34:32 [13946] Header: shib-authentication-instant: 2012-01-25T17:28:47.860Z
Jan 25 11:34:32 [13946] Header: shib-authentication-method: urn:oasis:names:tc:SAML:2.0:ac:classes:unspecified
Jan 25 11:34:32 [13946] Header: shib-identity-provider: https://###.umn.edu/idp/###
Jan 25 11:34:32 [13946] Header: dnt: 1
Jan 25 11:34:32 [13946] Header: cookie: testuser=###; umnAuthV2=###; _shibsession_###=_###
Jan 25 11:34:32 [13946] Header: accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

Jan 25 11:34:35 [14006] HTTP POST request, URL: /session/###
Jan 25 11:34:35 [14006] Header: accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Jan 25 11:34:35 [14006] Header: accept-language: en-us,en;q=0.5
Jan 25 11:34:35 [14006] Header: content-type: application/x-www-form-urlencoded
Jan 25 11:34:35 [14006] Header: user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1
Jan 25 11:34:35 [14006] Header: referer: https://###.umn.edu/session/###
Jan 25 11:34:35 [14006] Header: shib-session-id: _###
Jan 25 11:34:35 [14006] Header: host: localhost
Jan 25 11:34:35 [14006] Header: shib-authncontext-class: urn:oasis:names:tc:SAML:2.0:ac:classes:unspecified
Jan 25 11:34:35 [14006] Header: shib-session-index: ###
Jan 25 11:34:35 [14006] Header: accept-encoding: gzip, deflate
Jan 25 11:34:35 [14006] Header: shib-authentication-instant: 2012-01-25T17:28:47.860Z
Jan 25 11:34:35 [14006] Header: shib-authentication-method: urn:oasis:names:tc:SAML:2.0:ac:classes:unspecified
Jan 25 11:34:35 [14006] Header: shib-identity-provider: https://###.shib.umn.edu/idp/###
Jan 25 11:34:35 [14006] Header: cookie: testuser=###; umnAuthV2=###; _shibsession_###=_###
Jan 25 11:34:35 [14006] Header: accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Jan 25 11:34:35 [14006] Invalid https request: POST /session/###

What you can see from the above is that the following headers ARE NO LONGER PRESENT AS THEY SHOULD BE (while additional Shib-related headers do get added as seen above):

GET:
x-forwarded-server: #####.umn.edu
connection: Keep-Alive
x-forwarded-host: #####.umn.edu
x-forwarded-for: 63.231.xxx.yyy

POST:

x-forwarded-server: ###.umn.edu
connection: Keep-Alive
content-length: 54
x-forwarded-host: ###.umn.edu
x-forwarded-for: 63.231.xxx.yyy

It is the lack of the 'content-length' header in particular that our application complains about on the POST request.  However, the application will also be needing additional Shib-specific headers (locally-defined attributes) that are also not getting passed.  Additionally, the x-forwwarded-* headers go missing as well as the 'connection' header.  What could be stripping these headers upon turning on ShibUseHeaders?  I assume ShibSP is doing this.

Does anyone run a proxy that is Shib-protected?  Are you using ShibUseHeaders?  We are running Apache 2.2 and ShibSP 2.4.3.

Thanks in advance for any assistance you can provide!

[Cantor, Scott]

> It is the lack of the 'content-length' header in particular that our application
> complains about on the POST request. However, the application will also be
> needing additional Shib-specific headers (locally-defined attributes) that are
> also not getting passed. Additionally, the x-forwwarded-* headers go
> missing as well as the 'connection' header. What could be stripping these
> headers upon turning on ShibUseHeaders? I assume ShibSP is doing this.

It's not. The SP has no involvement in this whatsoever. It sets the request headers on the Apache request object. After that, anything that happens is the Apache handler that actually serves the request.

> Does anyone run a proxy that is Shib-protected? Are you using
> ShibUseHeaders? We are running Apache 2.2 and ShibSP 2.4.3.

I believe we have one in front of a Tomcat server, but I don't know if any of those headers are "missing" or any other details, just that they did it. I don't know if the proxy end needs the headers option or if it just sets them for the proxied hop directly based on environment variables.

-- Scott

--
To unsubscribe from this list send an email to users-un...@shibboleth.net

[Steve Siirila]

On Wed, Jan 25, 2012 at 5:36 PM, Cantor, Scott <cant...@osu.edu> wrote:

> It is the lack of the 'content-length' header in particular that our application
> complains about on the POST request.  However, the application will also be
> needing additional Shib-specific headers (locally-defined attributes) that are
> also not getting passed.  Additionally, the x-forwwarded-* headers go
> missing as well as the 'connection' header.  What could be stripping these
> headers upon turning on ShibUseHeaders?  I assume ShibSP is doing this.

It's not. The SP has no involvement in this whatsoever. It sets the request headers on the Apache request object. After that, anything that happens is the Apache handler that actually serves the request.

Sorry, I specifically meant the Apache mod_shib module -- which is bundled in my "ShibSP" software.

> Does anyone run a proxy that is Shib-protected?  Are you using
> ShibUseHeaders?  We are running Apache 2.2 and ShibSP 2.4.3.

I believe we have one in front of a Tomcat server, but I don't know if any of those headers are "missing" or any other details, just that they did it. I don't know if the proxy end needs the headers option or if it just sets them for the proxied hop directly based on environment variables.

Care to share your Apache configuration -- in particular, the Shib- and Proxy-related lines?  Here's what I'm using in case anyone sees something obviously wrong (I've pared it down a lot):

Listen 160.94.xxx.yyy:80
Listen 160.94.xxx.yyy:443
Listen 160.94.xxx.yyy:4443

LoadModule log_config_module libexec/mod_log_config.so
LoadModule ssl_module libexec/mod_ssl.so
LoadModule mod_shib /opt/shibboleth-sp/lib/shibboleth/mod_shib_22.so
LoadModule rewrite_module libexec/mod_rewrite.so
LoadModule proxy_module libexec/mod_proxy.so
LoadModule proxy_http_module libexec/mod_proxy_http.so

SSLProxyEngine On
ProxyPass / https://localhost/
ProxyPassReverse / https://localhost

<Location />
    AuthType shibboleth
    ShibRequestSetting requireSession 1
    ShibUseHeaders On
    require valid-user
</Location>

ServerTokens Full
ServerSignature On

SSLRandomSeed connect builtin
SSLRandomSeed startup file:/dev/urandom 512
SSLPassPhraseDialog  builtin
SSLSessionCache        shmcb:/var/opt/apache/...
SSLSessionCacheTimeout  300
SSLMutex  file:/var/opt/apache/...
SSLEngine on
SSLCipherSuite HIGH:MEDIUM:!SSLv2:!ADH:!NULL
SSLCertificateFile /etc/opt/apache/.../srv.crt
SSLCertificateKeyFile /etc/opt/apache/.../srv.key

<VirtualHost _default_:80>
    CustomLog /var/opt/apache/.../access-80 combined
    ErrorLog /var/opt/apache/.../errors-80
    SSLEngine off
    RewriteEngine On
    RewriteRule ^/ https://###.umn.edu [L,R]
</VirtualHost>

<VirtualHost _default_:4443>
    CustomLog /var/opt/apache/.../access-4443 combined
    ErrorLog /var/opt/apache/.../errors-4443
    SSLEngine on
</VirtualHost>

[Cantor, Scott]

On 1/25/12 7:32 PM, "Steve Siirila" <s...@umn.edu> wrote:
>
>Sorry, I specifically meant the Apache mod_shib module -- which is
>bundled in my "ShibSP" software.

Which is the SP, and again, has nothing to do with your issue. There is no
code in it that will remove headers that don't overlap with the ones it
controls.

Whether the behavior is observed without the SP or not, it is not the SP
doing it.

>Care to share your Apache configuration -- in particular, the Shib- and
>Proxy-related lines?

It isn't mine and I have no access to it. I simply note the fact that it's
been done.

[Steve Siirila]

Just a quick followup...  the issue was in the application which was silently discarding all header variables upon reaching one without a value.  And apparently either the Shib SP or the proxy likes sending value-less header variables (e.g. "Shib-AuthnContent-Decl:") the moment ShibUseHeaders got turned on.  Now that the application ignores these and reads the rest it eventually gets to the last few, including the critical one (Content-Length:).

(I hadn't gotten back to this in the past two weeks or I'd have realized this sooner.)

[Cantor, Scott]

> Just a quick followup... the issue was in the application which was silently
> discarding all header variables upon reaching one without a value. And
> apparently either the Shib SP or the proxy likes sending value-less header
> variables (e.g. "Shib-AuthnContent-Decl:") the moment ShibUseHeaders got
> turned on. Now that the application ignores these and reads the rest it
> eventually gets to the last few, including the critical one (Content-Length:).

The SP is doing that, because if it didn't, values sent by the client would spoof any headers you want to rely on. The empty ones are "cleared".

[Steve Siirila]

On Wed, Feb 8, 2012 at 3:44 PM, Cantor, Scott <cant...@osu.edu> wrote:

> Just a quick followup...  the issue was in the application which was silently
> discarding all header variables upon reaching one without a value.  And
> apparently either the Shib SP or the proxy likes sending value-less header
> variables (e.g. "Shib-AuthnContent-Decl:") the moment ShibUseHeaders got
> turned on.  Now that the application ignores these and reads the rest it
> eventually gets to the last few, including the critical one (Content-Length:).

The SP is doing that, because if it didn't, values sent by the client would spoof any headers you want to rely on. The empty ones are "cleared".

That certainly makes sense!  Thanks for pushing me in the right direction earlier (i.e. keeping me from barking up the wrong tree).