Apache proxy and mod_wsgi-express

89 views
Skip to first unread message

rmoo...@gmx.net

unread,
Apr 1, 2020, 3:46:21 PM4/1/20
to modwsgi
I have a frontend Apache that acts as a proxy to my backend mod_wsgi-express proxy to fix an issue that I had before (see https://groups.google.com/forum/#!topic/modwsgi/GTjMMm4wZJI).

My front end configuration looks like this:
<IfModule mod_ssl.c>
        <VirtualHost _default_:443>
                ServerName example.com
                ServerAdmin m...@example.com

                <Proxy *>
                  AuthType Basic
                  AuthUserFile /data/passwords/passwords.pwd
                  AuthName "myDemoApp"
                  <RequireAll>
                    Require valid-user
                  </RequireAll>
                </proxy>

                <Location "/">
                  ProxyPass              "http://example.com:8001/"
                  ProxyPassReverse       "http://example.com:8001/"
                  RequestHeader set X-Forwarded-Proto "https"
                </Location>

                :

        </VirtualHost>
</IfModule>

and my mod_wsgi-express looks like this:
mod_wsgi-express setup-server /data/website/myApp/wsgi.py \
    --reload-on-changes \
    --port 8001 \
    --user www-data --group www-data \
    --server-root /var/www/my_app \
    --log-level info \
    --access-log

This setup works perfectly fine and opens my page when I go on https://example.com. But I would like to use my proxy to forward the requests to different backends, depending on the URL.

To this end, I changed the <Location "/"> to <Location "/bla/">. Not when I open the website at https://example.com I get the Apache2 Default Page, which is what I would expect. But when I try to open https://example.com/bla/ it simply doesn't load the website (no error or something, just the Loading...). I don't get any error, neither in the frontend nor in the backend log.

Does anybody have an idea what might be wrong with my configuration?

Graham Dumpleton

unread,
Apr 1, 2020, 7:36:03 PM4/1/20
to mod...@googlegroups.com
Usually you wouldn't put ProxyPass inside of Location. What happens when you don't use Location and instead use:

                  ProxyPass  /bla            "http://example.com:8001/"
                  ProxyPassReverse /bla      "http://example.com:8001/"

Note how the sub URL is the first argument to ProxyPass.

For more details related to proxying to mod_wsgi-express see:


--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/889266ce-91e2-4ac3-a9ed-0fd13e6d9ed6%40googlegroups.com.

rmoo...@gmx.net

unread,
Apr 2, 2020, 2:49:03 AM4/2/20
to modwsgi
Thank you Graham, me setup is based on the blog posts you suggested. I started out with the ProxyPass not inside of Location and then tried all kinds of combinations, with no success. I had tried the following:

                ProxyPass               /bla "http://srv-lab-t-495.zhaw.ch:8001/"
                ProxyPassReverse        /bla "http://srv-lab-t-495.zhaw.ch:8001/"
                RequestHeader set X-Forwarded-Proto "https"
 
Any other ideas on what the issue could be?
To unsubscribe from this group and stop receiving emails from it, send an email to mod...@googlegroups.com.

Graham Dumpleton

unread,
Apr 2, 2020, 3:45:09 AM4/2/20
to mod...@googlegroups.com
So what happens when you access the backend directly?

To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/f5461cdd-e62a-4764-9216-ef7bf47da5e8%40googlegroups.com.

rmoo...@gmx.net

unread,
Apr 2, 2020, 3:59:51 AM4/2/20
to modwsgi
If I open the port on my firewall and access it directly at port 8001 (http://example.com:8001) with it works perfectly fine.

Am Donnerstag, 2. April 2020 09:45:09 UTC+2 schrieb Graham Dumpleton:
So what happens when you access the backend directly?
On 2 Apr 2020, at 5:49 pm, rmoo...@gmx.net wrote:

Thank you Graham, me setup is based on the blog posts you suggested. I started out with the ProxyPass not inside of Location and then tried all kinds of combinations, with no success. I had tried the following:

                ProxyPass               /bla "http://example.com:8001/"
                ProxyPassReverse        /bla "http://example.com:8001/"
                RequestHeader set X-Forwarded-Proto "https"
 

Graham Dumpleton

unread,
Apr 2, 2020, 4:09:52 AM4/2/20
to mod...@googlegroups.com
But with if you get on the host where front end Apache is running and you use 'curl' to access it using exact URL ProxyPass is using?

When you do access it via the Apache front end, with --log-level debug, does it log anything at all indicating the request is received by mod_wsgi-express?

To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/218165cc-50d2-4f2b-8989-9167b3127a6f%40googlegroups.com.

Raphael Monstein

unread,
Apr 2, 2020, 6:01:07 AM4/2/20
to mod...@googlegroups.com
Ah, I see. here's what I get from curl http://example.com:8001:
 
<!DOCTYPE html>
<html>
    <head>
        <meta http-equiv="X-UA-Compatible" content="IE=edge">
      <meta charset="UTF-8">
        <title>My title</title>
        <link rel="icon" type="image/x-icon" href="/_favicon.ico?v=1.4.1">
        <link rel="stylesheet" href="https://codepen.io/chriddyp/pen/bWLwgP.css">
<link rel="stylesheet" href="/assets/style.css">
<link rel="stylesheet" href="/assets/style.css?m=1585731972.8701963">
    </head>
    <body>

<div id="react-entry-point">
    <div class="_dash-loading">
        Loading...
    </div>
</div>

        <footer>
            <script id="_dash-config" type="application/json">{"url_base_pathname": null, "requests_pathname_prefix": "/", "ui": false, "props_check": false, "show_undo_redo": false}</script>
            <script src="/_dash-component-suites/dash_renderer/re...@16.8.6.min.js?v=1.1.2&m=1576595738"></script>
<script src="/_dash-component-suites/dash_renderer/reac...@16.8.6.min.js?v=1.1.2&m=1576595738"></script>
<script src="/_dash-component-suites/dash_renderer/prop-...@15.7.2.min.js?v=1.1.2&m=1576595738"></script>
<script src="/_dash-component-suites/dash_core_components/plotly-1.50.1.min.js?v=1.3.1&m=1576595950"></script>
<script src="/_dash-component-suites/dash_core_components/highlight.pack.js?v=1.3.1&m=1576595950"></script>
<script src="/_dash-component-suites/dash_core_components/dash_core_components.min.js?v=1.3.1&m=1576595950"></script>
<script src="/_dash-component-suites/dash_html_components/dash_html_components.min.js?v=1.0.1&m=1576596177"></script>
<script src="/_dash-component-suites/dash_renderer/dash_renderer.min.js?v=1.1.2&m=1576595738"></script>
            <script id="_dash-renderer" type="application/javascript">var renderer = new DashRenderer();</script>
        </footer>
    </body>
</html>

And this what I get from curl -u user:password https://example.com/bla:
<!DOCTYPE html>
<html>
    <head>
        <meta http-equiv="X-UA-Compatible" content="IE=edge">
      <meta charset="UTF-8">
        <title>My title</title>
        <link rel="icon" type="image/x-icon" href="/_favicon.ico?v=1.4.1">
        <link rel="stylesheet" href="https://codepen.io/chriddyp/pen/bWLwgP.css">
<link rel="stylesheet" href="/assets/style.css">
<link rel="stylesheet" href="/assets/style.css?m=1585731972.8701963">
    </head>
    <body>

<div id="react-entry-point">
    <div class="_dash-loading">
        Loading...
    </div>
</div>

        <footer>
            <script id="_dash-config" type="application/json">{"url_base_pathname": null, "requests_pathname_prefix": "/", "ui": false, "props_check": false, "show_undo_redo": false}</script>
            <script src="/_dash-component-suites/dash_renderer/re...@16.8.6.min.js?v=1.1.2&m=1576595738"></script>
<script src="/_dash-component-suites/dash_renderer/reac...@16.8.6.min.js?v=1.1.2&m=1576595738"></script>
<script src="/_dash-component-suites/dash_renderer/prop-...@15.7.2.min.js?v=1.1.2&m=1576595738"></script>
<script src="/_dash-component-suites/dash_core_components/plotly-1.50.1.min.js?v=1.3.1&m=1576595950"></script>
<script src="/_dash-component-suites/dash_core_components/highlight.pack.js?v=1.3.1&m=1576595950"></script>
<script src="/_dash-component-suites/dash_core_components/dash_core_components.min.js?v=1.3.1&m=1576595950"></script>
<script src="/_dash-component-suites/dash_html_components/dash_html_components.min.js?v=1.0.1&m=1576596177"></script>
<script src="/_dash-component-suites/dash_renderer/dash_renderer.min.js?v=1.1.2&m=1576595738"></script>
            <script id="_dash-renderer" type="application/javascript">var renderer = new DashRenderer();</script>
        </footer>
    </body>
</html>

The both look pretty much the same to me, except that if I access it with the browser it gets stuck at "Loading..." when I go via the proxy.
 
I set the log level to debug on the backend server and that works (I can see some debug messages in the error log from the startup). When I do curl http://example.com:8001 I get the following:
 
[Thu Apr 02 10:50:17.240939 2020] [authz_core:debug] [pid 11192:tid 139984647812864] mod_authz_core.c(820): [client 127.0.0.1:39026] AH01626: authorization result of <RequireAny>: granted
[Thu Apr 02 10:50:17.241057 2020] [authz_core:debug] [pid 11192:tid 139984647812864] mod_authz_core.c(820): [client 127.0.0.1:39026] AH01626: authorization result of Require all granted: granted
[Thu Apr 02 10:50:17.241070 2020] [authz_core:debug] [pid 11192:tid 139984647812864] mod_authz_core.c(820): [client 127.0.0.1:39026] AH01626: authorization result of <RequireAny>: granted

But when I do curl -u user:password https://example.com/bla I don't get a new entry in the log file.
 
Do you know what that means and how I can fix it?
 
 
 
Gesendet: Donnerstag, 02. April 2020 um 10:09 Uhr
Von: "Graham Dumpleton" <graham.d...@gmail.com>
An: mod...@googlegroups.com
Betreff: Re: [modwsgi] Apache proxy and mod_wsgi-express

Graham Dumpleton

unread,
Apr 2, 2020, 6:53:58 PM4/2/20
to mod...@googlegroups.com
Try using the --trust-??? options to mod_wsgi-express mentioned in:


The fact that it is saying "Loading ..." suggests that that would be filled in later by some Javaascript code.

If your application is not handling the X-Forwarded-??? headers, it may be generating an incorrect URL in a response that Javascript code is using to do further AJAX requests to something.

Also try using the option:

   --mount-point /bla

option to mod_wsgi-express so the backend mount points matches the front end mount point. If don't do this, the backend will think it is mounted at the root of the site, which again may be mucking up generated URLs.

rmoo...@gmx.net

unread,
Apr 3, 2020, 8:20:07 AM4/3/20
to modwsgi
I have been again trying all kinds of combinations but none of them work. To clear up things, I have made two versions, one where the entry point is "/" and the other is "/bla". Of course, only one of the is active at the same time. This way I can check that I didn't break anything while playing around. When my entry point is "/", it still works fine. My front end config looks like this:
# config "/"
  #ProxyPass               / http://example.com:8001/
  #ProxyPassReverse        / http://example.com:8001/

# config "/bla"
  ProxyPass               /bla http://example.com:8001/bla
  ProxyPassReverse        /bla http://example.com:8001/bla

# general
  RequestHeader set X-Forwarded-Port 443
  RequestHeader set X-Forwarded-Scheme https

And my mod_wsgi-express is set up in the following way for entry point "/":
mod_wsgi-express setup-server /data/website/myApp/wsgi.py \
    --reload-on-changes \
    --port 8001 \
    --user www-data --group www-data \
    --server-root /var/www/my_app \
    --log-level debug \
    --access-log \
    --trust-proxy-header X-Forwarded-Host \
    --trust-proxy-header X-Forwarded-Port \
    --trust-proxy-header X-Forwarded-For \
    --trust-proxy-header X-Forwarded-Scheme

And for the entry point "/bla":
mod_wsgi-express setup-server /data/website/myApp/wsgi.py \
    --reload-on-changes \
    --port 8001 \
    --user www-data --group www-data \
    --server-root /var/www/my_app \
    --log-level debug \
    --access-log \
    --trust-proxy-header X-Forwarded-Host \
    --trust-proxy-header X-Forwarded-Port \
    --trust-proxy-header X-Forwarded-For \
    --trust-proxy-header X-Forwarded-Scheme \
    --mount-point /bla

This seems to be the most reasonable configuration and in my world this should work, but it doesn't (I also found this that seems to work and is very similar to what I do).

If I understood your blog posts correctly, I don't have to change anything in my Flask application for this to work. Is that correct?

Any ideas what might be wrong here?
<script src="/_dash-component-suites/dash_renderer/react-dom@16.8.6.min.js?v=1.1.2&m=1576595738"></script>
<script src="/_dash-component-suites/dash_renderer/prop-types@15.7.2.min.js?v=1.1.2&m=1576595738"></script>
<script src="/_dash-component-suites/dash_renderer/react-dom@16.8.6.min.js?v=1.1.2&m=1576595738"></script>
<script src="/_dash-component-suites/dash_renderer/prop-types@15.7.2.min.js?v=1.1.2&m=1576595738"></script>

<script src="/_dash-component-suites/dash_core_components/plotly-1.50.1.min.js?v=1.3.1&m=1576595950"></script>
<script src="/_dash-component-suites/dash_core_components/highlight.pack.js?v=1.3.1&m=1576595950"></script>
<script src="/_dash-component-suites/dash_core_components/dash_core_components.min.js?v=1.3.1&m=1576595950"></script>
<script src="/_dash-component-suites/dash_html_components/dash_html_components.min.js?v=1.0.1&m=1576596177"></script>
<script src="/_dash-component-suites/dash_renderer/dash_renderer.min.js?v=1.1.2&m=1576595738"></script>
            <script id="_dash-renderer" type="application/javascript">var renderer = new DashRenderer();</script>
        </footer>
    </body>
</html>

The both look pretty much the same to me, except that if I access it with the browser it gets stuck at "Loading..." when I go via the proxy.
 
I set the log level to debug on the backend server and that works (I can see some debug messages in the error log from the startup). When I do curl http://example.com:8001 I get the following:
 
[Thu Apr 02 10:50:17.240939 2020] [authz_core:debug] [pid 11192:tid 139984647812864] mod_authz_core.c(820): [client 127.0.0.1:39026] AH01626: authorization result of <RequireAny>: granted
[Thu Apr 02 10:50:17.241057 2020] [authz_core:debug] [pid 11192:tid 139984647812864] mod_authz_core.c(820): [client 127.0.0.1:39026] AH01626: authorization result of Require all granted: granted
[Thu Apr 02 10:50:17.241070 2020] [authz_core:debug] [pid 11192:tid 139984647812864] mod_authz_core.c(820): [client 127.0.0.1:39026] AH01626: authorization result of <RequireAny>: granted

But when I do curl -u user:password https://example.com/bla I don't get a new entry in the log file.
 
Do you know what that means and how I can fix it?
 
Gesendet: Donnerstag, 02. April 2020 um 10:09 Uhr
Von: "Graham Dumpleton" <graham....@gmail.com>
To unsubscribe from this group and stop receiving emails from it, send an email to mod...@googlegroups.com.
 
--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/trinity-16397838-77af-4d30-93a8-8e9ab4fbf39d-1585821663425%403c-app-gmx-bap75.

Graham Dumpleton

unread,
Apr 3, 2020, 4:18:16 PM4/3/20
to mod...@googlegroups.com
Try also:

--mount-point /bla/

I can't remember if the trailing slash can make a difference.

To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/85fbfdb4-f239-4309-95b2-7f998bb9efe1%40googlegroups.com.

Graham Dumpleton

unread,
Apr 3, 2020, 4:44:19 PM4/3/20
to mod...@googlegroups.com
BTW, also suggest trying:


Luckily in mod_wsgi-express this is easy and you don't need to integrate it yourself.

Add options:

    --enable-recorder --recorder-directory /tmp/wsgi-requests

Not sure if target directory needs to exist first.

It will dump files with details of any requests received so you can check what details are sent in WSGI request environ, but also in response headers and content, so can compare.

Raphael Monstein

unread,
Apr 7, 2020, 2:06:39 PM4/7/20
to mod...@googlegroups.com
Graham, thank you for the help. Your hints were very helpful in figuring out what is going on here (and learn quite a bit as a side-effect). I finally managed to find a configuration that works for us. I'd like to post it here in case somebody else is in a similar situation.
 
The mod_wsgi-express is set up in the following way:
mod_wsgi-express setup-server /data/website/myApp/wsgi.py \
    --reload-on-changes \
    --port 8001 \
    --user www-data \
    --group www-data \
    --server-root /var/www/my_app \
    --access-log \
    --mount-point /bla \

    --trust-proxy-header X-Forwarded-Host \
    --trust-proxy-header X-Forwarded-Port \
    --trust-proxy-header X-Forwarded-For \
    --trust-proxy-header X-Forwarded-Scheme

and the relevant bit of my front end apache configuration is this:
<VirtualHost _default_:443>
  [...]
  <Proxy *>

  AuthType Basic
  AuthUserFile /data/passwords/passwords.pwd
  AuthName "myDemoApp"
  <RequireAll>
    Require valid-user
  </RequireAll>
  </proxy>

  ProxyPass /bla http://example.com:8001/bla
  ProxyHTMLURLMap http://example.com:8001 /bla

  <Location /bla/>
    ProxyPassReverse       /
    RequestHeader set X-Forwarded-Proto "https"
    RequestHeader set X-Forwarded-Port  443
    ProxyHTMLEnable On
    ProxyHTMLExtended On
    ProxyHTMLURLMap / /bla/
    RequestHeader unset Accept-Encoding
  </Location>
  [...]
</VirtualHost>
The key for me was to use mod_proxy_html (ProxyHTMLEnable, ProxyHTMLExtended, ProxyHTMLExtended, and ProxyHTMLURLMap).
 
@Graham: Do you know, by any chance, a way how to avoid using mod_proxy_html in his situation?
I don't feel too comfortable with this solution, since I had to set ProxyHTMLExtended On. As pointed out in the documentation, this is not ideal and can cause problems.
 
 
 
Gesendet: Freitag, 03. April 2020 um 22:44 Uhr
Von: "Graham Dumpleton" <graham.d...@gmail.com>

An: mod...@googlegroups.com
Betreff: Re: [modwsgi] Apache proxy and mod_wsgi-express
Try also:
 
--mount-point /bla/
 
<script src="/_dash-component-suites/dash_renderer/reac...@16.8.6.min.js?v=1.1.2&m=1576595738"></script>
<script src="/_dash-component-suites/dash_renderer/prop-...@15.7.2.min.js?v=1.1.2&m=1576595738"></script>
<script src="/_dash-component-suites/dash_renderer/reac...@16.8.6.min.js?v=1.1.2&m=1576595738"></script>
<script src="/_dash-component-suites/dash_renderer/prop-...@15.7.2.min.js?v=1.1.2&m=1576595738"></script>

Graham Dumpleton

unread,
Apr 7, 2020, 5:34:43 PM4/7/20
to mod...@googlegroups.com
What Python web framework are you using?

That you need to do this indicates you aren't constructing URLs in responses correctly. Python web frameworks usually have a helper function to do this for you. They will use SCRIPT_NAME from the WSGI environ as mount point and add that in front. This was why said to try with --mount-point to match what used on front end, so frameworks could do this properly.

If not using a known Python web framework which has such helpers and doing raw WSGI, you need to do this yourself. See:


In other words, you shouldn't be using absolute URL paths in your responses, and either calculate it (using helper from function in web framework), or use relative paths if is simple use case.

Graham

rmoo...@gmx.net

unread,
Apr 8, 2020, 5:27:14 AM4/8/20
to modwsgi
I'm using Flask and Dash for the app.

Since I spent too much time on this already, I'll keep the configuration as is it. Hopefully, I will fix this properly in the future, but at the moment I don't have the time. If anybody as a decent tutorial that describes how to deal with this issue, I would greatly appreciate a link to it.
Graham

Von: "Graham Dumpleton" <graham....@gmail.com>
An: mod...@googlegroups.com
Betreff: Re: [modwsgi] Apache proxy and mod_wsgi-express
Try also:
 
--mount-point /bla/
 
<script src="/_dash-component-suites/dash_renderer/react-d...@16.8.6.min.js?v=1.1.2&m=1576595738"></script>
<script src="/_dash-component-suites/dash_renderer/prop-ty...@15.7.2.min.js?v=1.1.2&m=1576595738"></script>
<script src="/_dash-component-suites/dash_renderer/react-d...@16.8.6.min.js?v=1.1.2&m=1576595738"></script>
<script src="/_dash-component-suites/dash_renderer/prop-ty...@15.7.2.min.js?v=1.1.2&m=1576595738"></script>
 
--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/trinity-2f93db63-8850-4113-8a0e-da0071bc4a8c-1586282795652%403c-app-gmx-bs67.

Graham Dumpleton

unread,
Apr 8, 2020, 5:30:41 AM4/8/20
to mod...@googlegroups.com
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/eba35b71-8ad8-4ffb-b6b8-3ea601478123%40googlegroups.com.

Graham Dumpleton

unread,
Apr 8, 2020, 5:31:49 AM4/8/20
to mod...@googlegroups.com
In particular, item 5 in the list of why it exists says:

    If your application is placed outside the URL root, for example, in /myapplication instead of /, url_for() properly handles that for you.
Reply all
Reply to author
Forward
0 new messages