HTTPS problem with Django, mod_wsgi and Apache

[Jennifer Mehl]

Hi there,

I am backpedalling a bit from my previous attempt to chroot mod_wsgi - instead, for now, just to get this Django application running, for simplicity, I am going to start out with just running it as a daemon as a restricted user.

In doing the final testing of my application on various browsers, I have noticed some strange problems.  

When I run Django/mod_wsgi/Apache on port 80 (same config as below, minus the mod_ssl stuff)  or use the django development runserver 0.0.0.0:80, and disable the following settings in settings.py (#SESSION_COOKIE_SECURE = True #CSRF_COOKIE_SECURE = True) these browsers work correctly in the app.

However, when running Django application running through mod_wsgi and HTTPS/port 443 in Apache, I see problems with both IE and Safari browsers.  After login on Internet Explorer, page timeouts occur in various locations, reporting "This page can't be displayed".  On Safari, the app won't get past the secondary Duo MFA authentication step, saying "Server unexpectedly dropped the connection." It is not a consistent behavior - seems to happen more frequently if I click quickly through links.   Sometimes if I wait long enough to click, it might work momentarily, but then not again a moment later.  This behavior does NOT happen using Chrome or Firefox browsers on any OS.

Apache config:

<IfModule mod_ssl.c>

<VirtualHost *:443>

    ServerName **redacted**

#Django WSGI - Daemon

        WSGIScriptAlias / /var/www/transfergateway/myproject/apache/wsgi.py

        WSGIProcessGroup file-xfer 

        WSGIDaemonProcess file-xfer user=mod_wsgi group=mod_wsgi processes=2 threads=25 python-path=/var/www/transfergateway

        

<Directory /var/www/transfergateway/myproject/apache>

<Files wsgi.py>

Order deny,allow

Allow from all

</Files>

</Directory>

Alias /robots.txt /var/www/transfergateway/myproject/myapp/static/robots.txt

Alias /favicon.ico /var/www/transfergateway/myproject/myapp/static/favicon.ico

AliasMatch ^/([^/]*\.css) /var/www/transfergateway/myproject/myapp/static/styles/$1

Alias /media/ /var/www/transfergateway/myproject/myapp/media/

Alias /static/ /var/www/transfergateway/myproject/myapp/static/

<Directory /var/www/transfergateway/myproject/myapp/static>

Order deny,allow

Allow from all

</Directory>

<Directory /var/www/transfergateway/myproject/myapp/media>

Order deny,allow

Allow from all

</Directory>

    ErrorLog ${APACHE_LOG_DIR}/error.log

    CustomLog ${APACHE_LOG_DIR}/access.log combined

    SSLEngine on

    SSLCertificateFile /etc/ssl/certs/***

    SSLCertificateKeyFile /etc/ssl/private/**

    SSLCertificateChainFile /etc/ssl/certs/**

    SSLCipherSuite HIGH:!aNULL:!MD5

</VirtualHost>

</IfModule>

So, I'm concluding that the HTTPS problem is one of two things: how I am configuring mod_wsgi with HTTPS, or some issue inside the Django code (but HTTPS works on some browsers with no issues, so I'm stumped...)

Is there anything special that I need to do in mod_wsgi or the Django application itself, in order to make the application HTTPS only?  (I am not a Python or Django developer, so I would be passing info on to the actual application developer for resolution.)  Any ideas?

thank you,

Jennifer

[Graham Dumpleton]

I'll go through the description you gave me and see if can suggest anything, but first up, what version of mod_wsgi are you using?

If you are using mod_wsgi 4.4.0 make sure you update to 4.4.1. The newer version resolves a potential for process crashing introduced in 4.4.0.

Graham

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To post to this group, send email to mod...@googlegroups.com.
Visit this group at http://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.

[Graham Dumpleton]

One more question. What version of Apache are you using?

If you are stuck on a quite old Apache 2.2.X version that would be a concern as there were various SSL related issues patched during the life of Apache 2.2.X.

Graham

[Jennifer Mehl]

I’m on the latest for Ubuntu 14.04LTS - 2.4.7-1ubuntu4.1. I have been using the updated mod_wsgi3.4 from Ubuntu.

At this point I was thinking about trying my Django application in a different WSGI server to see if I can narrow down if the problem is with the Django code or something with mod_wsgi. I was thinking about uwsgi (trying to find something quick and easy to test) or nginx.

Again, the weird browser behavior I describe below only happens when using Apache/HTTPS, port 443, in mod_wsgi (not Apache/HTTP in mod_wsgi or the Django development server in port 80).

I’m kind of at my wit’s end trying to narrow down *where* the problem is (if it’s something in the Django code, I only have one more day until my developer leaves for a few weeks for winter break…) Do you think there any debugging I can do by looking at the developer console in the affected browsers - for instance comparing the affected pages on a working port 80 vs the same pages on the non-working SSL/port 443 connection?

thank you,
Jennifer

> You received this message because you are subscribed to a topic in the Google Groups "modwsgi" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/modwsgi/S1if2HhkGGE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to modwsgi+u...@googlegroups.com.

[Graham Dumpleton]

If you are using mod_wsgi 3.4 that could be a problem in itself.

Recent versions of Ubuntu as I understand it use Apache 2.4, but such an old version of mod_wsgi may have issues on Apache 2.4. At the minimum would need to have mod_wsgi 3.5 from memory as some Apache 2.4 fixes were back ported to 3.5. It is unlikely they back ported those themselves to 3.4 for 14.04.

Either way, mod_wsgi itself shouldn't be causing any problems with HTTPS as it is Apache that deals with all that and mod_wsgi has nothing to do with the handling of secure connections. When mod_wsgi sees a request that came via HTTPS it sees it as being no different to a HTTP request with the exception of what the wsgi.url_scheme attribute is set to. It is therefore more likely to be an Apache configuration issue or issue with the code of Apache itself.

FWIW, mod_wsgi 3.4 means that Ubuntu version is almost 20 versions behind. Even Ubuntu 14.10 has only mod_wsgi 3.5. It is quite frustrating that they haven't been bothered to update their packages to more recent versions even if only for the most recent 14.10.

About the only thing I can suggest if it is readily reproducible, is to use request logging such as described in:

http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Tracking_Request_and_Response

to see if when a request has issues, that the WSGI application actually returned the requests properly.

If it isn't, then use something like:

http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Extracting_Python_Stack_Traces

to get out Python stack traces for where a request handler may be stuck.

Both can be fiddly so sounds like you aren't going to have time to do that.

Graham

[Jennifer Mehl]

Thanks for this info. I’ll try a newer mod_wsgi.

It’s very odd to me that the app works fine in mod_wsgi/Apache with no SSL but parts become broken in certain browsers once SSL is enabled.

At any rate, thanks for the guidance and I’ll report back if I find a fix!

—Jennifer

[Graham Dumpleton]

You will unfortunately not find a binary OS supplied Ubuntu 10.4 package for mod_wsgi which is newer.

Your only choice would be to compile from source code.

Graham

[Jennifer Mehl]

No problem, if I have to compile from source, then I will try that.

One last thing regarding HTTPS - how do I ensure that I have the wsgi.url_scheme set correctly?

Here is my wsgi.py file:

import os
import sys

path='/var/www/transfergateway/myproject'

#if path not in sys.path:
#sys.path.append(path)

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")

#HTTPS
os.environ['HTTPS'] = "on"

# This application object is used by any WSGI server configured to use this
# file. This includes Django's development server, if the WSGI_APPLICATION
# setting points here.
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()




and here is relevant stuff from my settings.py file:

import os
PROJECT_ROOT = os.path.realpath(os.path.dirname(__file__))


#turn off debug when going to production
DEBUG = "FALSE"
TEMPLATE_DEBUG = DEBUG


# Python dotted path to the WSGI application used by Django's runserver.
WSGI_APPLICATION = 'myproject.wsgi.application'


#session expire at browser close
SESSION_EXPIRE_AT_BROWSER_CLOSE = True
SESSION_COOKIE_HTTPONLY=True

#idle timeout
SESSION_IDLE_TIMEOUT=900

#HTTPS stuff - secure proxy SSL header - do I need this?
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
#HTTPS stuff - secure cookies
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True

#HTTPS WSGI
os.environ['wsgi.url_scheme'] = 'https'

[Graham Dumpleton]

Hmmm, this looks really dangerous:

DEBUG = "FALSE"

The DEBUG setting is meant to be a boolean value, not a string.

Because you are setting it to a non empty string, it will be interpreted as True and so you have debug mode enabled.

That is not good as sensitive information could be exposed back to users in error pages shown in the browser.

Running in debug mode might cause other issues as well.

Ensure you are setting it to:

DEBUG = False

Also, setting:

os.environ['HTTPS'] = "on"
os.environ['wsgi.url_scheme'] = 'https'

will not do anything.

The wsgi.url_scheme is an attribute which is passed down by mod_wsgi in the details for each request. A web framework will use the flag from the request details. The main thing it controls is merely the construction of absolute URLs when needing to be added to response headers or maybe response content in some cases.

In other words, you do not need to set it and setting it in environment variables wouldn't do anything anyway.

Next, setting:

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")

is okay if you have just the one Django site, but be careful in using this if you are running more than one. Safer to use:

os.environ["DJANGO_SETTINGS_MODULE"] = "myproject.settings"

More details in:

http://blog.dscpl.com.au/2012/10/requests-running-in-wrong-django.html

You also don't need:

SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')

if Apache is your front facing web server. You would only need this if you had a further front end proxy such as nginx in front of Apache and nginx had been configured to actually introduce these headers. That your Apache is accepting HTTPS requests would indicate that you don't have an nginx in front.

Now as to determine whether wsgi.url_scheme is set properly, the easiest way is to take a copy of:

def application(environ, start_response):
status = '200 OK'
output = str(environ.get('wsgi.url_scheme'))

response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(output)))]
start_response(status, response_headers)

return [output]

Put it in a file called check.py nest to your existing wsgi.py file.

In the Apache configuration, BEFORE THE LINE:

WSGIScriptAlias / /var/www/transfergateway/myproject/apache/wsgi.py

add:

WSGIScriptAlias /wsgicheck /var/www/transfergateway/myproject/apache/check.py

Then down further where have:

<Directory /var/www/transfergateway/myproject/apache>
<Files wsgi.py>
Order deny,allow
Allow from all
</Files>
</Directory>

Change it to:

<Directory /var/www/transfergateway/myproject/apache>
<Files wsgi.py>
Order deny,allow
Allow from all
</Files>

<Files check.py>

Order deny,allow
Allow from all
</Files>
</Directory>

Restart Apache and then hit the URL of the site for /wsgicheck

You should see 'https' returned in the page.

Hope this helps.

Graham

[Jennifer Mehl]

Thanks - I will try these fixes!

Jennifer

[Jennifer Mehl]

Thank you. Good to get those things all cleaned up.

I also compiled and installed v4.4.1 of mod_wsgi from source and removed the 3.4 Ubuntu version from my system.

Setting DEBUG = False seems to break my application - I get a “Bad Request 400” error back in my browser - so I will check in with the developer on that one.

I’ve removed the extraneous environment variables and also the SSL proxy setting. I am only using mod_wsgi with Apache, so, as you say, it shouldn’t need that anyhow.

I’ve done the test for the /wsgicheck and it does return a value of https. Thanks for helping me verify that functionality.

So, this leaves me with looking at Apache as a culprit - or again, the Django code itself. It’s very odd how only the two browsers are showing issues and they are completely different issues…

thanks,
Jennifer

[Charles Yeomans]

I've used mod_wsgi for several years and had to install via a bash script (so that puppet could do it) to get current versions, but now one can install using pip. The only issue I found was that the pip install does not create mod_wsgi.load or mod_wsgi_conf, but those were simple enough to supply myself.


Charles Yeomans

[Graham Dumpleton]

It is on a long TODO list to generate Apache configuration file snippets for most basic of things where people don't want to use mod_wsgi-express from init scripts and instead want to integrate into the existing Apache configuration, even though they then loose out on the benefits of mod_wsgi-express.

Right now all that is available is once you do a pip install mod_wsgi, you can run:

$ mod_wsgi-express module-location
/Users/graham/Projects/mod_wsgi/venv/lib/python2.7/site-packages/mod_wsgi-4.4.1-py2.7-macosx-10.8-intel.egg/mod_wsgi/server/mod_wsgi-py27.so

so you can find out where pip installed the .so file in case you want to refer to it out of the Python installation when loading it into Apache. You might be able to use that at least to know from where to copy the .so file.

Alternatively, you can run:

$ sudo mod_wsgi-express install-module
LoadModule wsgi_module /usr/libexec/apache2/mod_wsgi-py27.so
WSGIPythonHome /Users/graham/Projects/mod_wsgi/venv

This will install the .so file into your Apache modules directory for you.

It will also give you the two key lines to stick in the Apache configuration, with the latter being what Python installation the module was compiled against.

Down the track, for Django at least, I have been contemplating a 'httpdconf' Django management command.

When this is run it would create and populate an 'apache' directory in your Django project with a WSGI script file and Apache configuration files snippets for loading and setting up the module, plus the specific snippets to mount your Django application and any static file assets. These snippets could be Include'd direct into the Apache configuration, copy pasted manually.

The reason for doing this as that so many people seem to still screw up the configuration even when following the Django documentation about mod_wsgi.

The other thing on the TODO list is have mod_wsgi-express generate init.d scripts for various Linux variants so that you can ignore the existing Apache and just start mod_wsgi-express on OS startup.

If you are not using Apache for anything else this would be better than relying on default Apache configuration which generally sucks for Python web applications. Better to use my curated mod_wsgi-express configuration.

Graham

[Jason Garber]

Hi Jennifer,

May I suggest you simplify your apache config by running apache on 127.0.0.1:8086 (for example) and placing nginx in front of it proxying requests to apache.  Use nginx for ssl termination.

It is dead simple and uncomplicates the apache config.

I can provide complete example configs if you wish.

Thanks!
Jason

[Jennifer Mehl]

Jason,

Having complete example configs would be fantastic. Turning on SSL in Apache is what is currently making parts of the app 'break' in IE and Safari. It would be great if I could rule out the application code - changing front end web servers is probably the only way to do that.

Thanks in advance for the help!

Jennifer

[Jason Garber]

I will send in the morning (eastern).

[Jennifer Mehl]

Fantastic - thanks again!

Jennifer

[Graham Dumpleton]

One question. Is:

SSLCertificateFile /etc/ssl/certs/***
SSLCertificateKeyFile /etc/ssl/private/**
SSLCertificateChainFile /etc/ssl/certs/**
SSLCipherSuite HIGH:!aNULL:!MD5

what you actually have in the Apache configuration, including the '**', or did you do that to mask information.

SSL configurations I have seen don't tend to have SSLCertificateChainFile either. Not sure if that is a requirement for you or not.

Generally I just use:

SLCertificateFile server.crt
SSLCertificateKeyFile server.key

I am always using self signed certificate files though.

Anyway, if you are happy with trying radical solutions, or at least validating Apache/mod_wsgi with SSL works with a configuration done by someone else, I have this dead horse called 'pip' I have been trying to sell with not much luck.

Seriously, for a perhaps quick way of testing an alternate SSL configuration with Apache, try this:

pip install mod_wsgi

sudo mod_wsgi-express start-server --user mod_wsgi --group mod_wsgi --port 80 --ssl-port 443 \
--ssl-certificate server --https-only --server-name redacted --home /var/www/transfergateway \
--url-alias /media /var/www/transfergateway/myproject/myapp/media \
--url-alias /static /var/www/transfergateway/myproject/myapp/static \
/var/www/transfergateway/myproject/apache/wsgi.py

You would need to stop the existing Apache first. Change 'redacted' to the actual ServerName value. The user and group to actual names for them. And have the SSL server.cert and server.key files together in the same directory and change 'server' argument to --ssl-certificate to be path to directory they are in, with the common base name part of the files on the end. That is, with extensions dropped off.

Okay, maybe too radical, but believe it or not that command line should hopefully run up Apache/mod_wsgi against your Django site if I got all the arguments right, with HTTPS all setup and in a HTTPS only configuration such that access to HTTP will redirect automatically to HTTPS URLs.

Worth a try I guess if you really get stuck. :-)

Graham

[Charles Yeomans]

I had only barely looked at the mod_wsgi-express stuff; I was in a bit of hurry when moving to mod_wsgi v4. But i looked at the source code last night, so I might have to give this a try.


Charles Yeomans

[Jennifer Mehl]

Hi there,

The *** in the configuration file are actually just redacted/masked information.

I tried removing the SSLCertificateChainFile and it had no effect, so I think it’s safe to leave it out. Ours are “real” certs issued by InCommon.

Thanks for the info using pip / mod_wsgi-express- I will give that a try before moving on to tying some request logging and then Nginx/other components.

—Jennifer

>>> Setting DEBUG False seems to break my application - I get a “Bad Request 400” error back in my browser - so I will check in with the developer on that one.

>>>
>>> I’ve removed the extraneous environment variables and also the SSL proxy setting. I am only using mod_wsgi with Apache, so, as you say, it shouldn’t need that anyhow.
>>>
>>> I’ve done the test for the /wsgicheck and it does return a value of https. Thanks for helping me verify that functionality.
>>>
>>> So, this leaves me with looking at Apache as a culprit - or again, the Django code itself. It’s very odd how only the two browsers are showing issues and they are completely different issues…
>>>
>>> thanks,
>>> Jennifer
>>>
>>>
>>>
>>>> On Dec 16, 2014, at 4:41 PM, Graham Dumpleton <graham.d...@gmail.com> wrote:
>>>>
>>>> Hmmm, this looks really dangerous:
>>>>

>>>> DEBUG "FALSE"

>>>>
>>>> The DEBUG setting is meant to be a boolean value, not a string.
>>>>
>>>> Because you are setting it to a non empty string, it will be interpreted as True and so you have debug mode enabled.
>>>>
>>>> That is not good as sensitive information could be exposed back to users in error pages shown in the browser.
>>>>
>>>> Running in debug mode might cause other issues as well.
>>>>
>>>> Ensure you are setting it to:
>>>>

>>>> DEBUG False
>>>>
>>>> Also, setting:
>>>>
>>>> os.environ['HTTPS'] "on"
>>>> os.environ['wsgi.url_scheme'] 'https'

>>>>
>>>> will not do anything.
>>>>
>>>> The wsgi.url_scheme is an attribute which is passed down by mod_wsgi in the details for each request. A web framework will use the flag from the request details. The main thing it controls is merely the construction of absolute URLs when needing to be added to response headers or maybe response content in some cases.
>>>>
>>>> In other words, you do not need to set it and setting it in environment variables wouldn't do anything anyway.
>>>>
>>>> Next, setting:
>>>>
>>>> os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")
>>>>
>>>> is okay if you have just the one Django site, but be careful in using this if you are running more than one. Safer to use:
>>>>

>>>> os.environ["DJANGO_SETTINGS_MODULE"] "myproject.settings"

>>>>
>>>> More details in:
>>>>
>>>> http://blog.dscpl.com.au/2012/10/requests-running-in-wrong-django.html
>>>>
>>>> You also don't need:
>>>>

>>>> SECURE_PROXY_SSL_HEADER ('HTTP_X_FORWARDED_PROTO', 'https')

>>>>
>>>> if Apache is your front facing web server. You would only need this if you had a further front end proxy such as nginx in front of Apache and nginx had been configured to actually introduce these headers. That your Apache is accepting HTTPS requests would indicate that you don't have an nginx in front.
>>>>
>>>> Now as to determine whether wsgi.url_scheme is set properly, the easiest way is to take a copy of:
>>>>
>>>> def application(environ, start_response):

>>>> status '200 OK'
>>>> output str(environ.get('wsgi.url_scheme'))
>>>>
>>>> response_headers [('Content-type', 'text/plain'),

>>>>> os.environ['HTTPS'] "on"

>>>>>
>>>>> # This application object is used by any WSGI server configured to use this
>>>>> # file. This includes Django's development server, if the WSGI_APPLICATION
>>>>> # setting points here.
>>>>> from django.core.wsgi import get_wsgi_application

>>>>> application get_wsgi_application()

>>>>>
>>>>>
>>>>>
>>>>>
>>>>> and here is relevant stuff from my settings.py file:
>>>>>
>>>>> import os

>>>>> PROJECT_ROOT os.path.realpath(os.path.dirname(__file__))

>>>>>
>>>>>
>>>>> #turn off debug when going to production

>>>>> DEBUG "FALSE"
>>>>> TEMPLATE_DEBUG DEBUG

>>>>>
>>>>>
>>>>> # Python dotted path to the WSGI application used by Django's runserver.

>>>>> WSGI_APPLICATION 'myproject.wsgi.application'

>>>>>
>>>>>
>>>>> #session expire at browser close

>>>>> SESSION_EXPIRE_AT_BROWSER_CLOSE True
>>>>> SESSION_COOKIE_HTTPONLY=True
>>>>>
>>>>> #idle timeout
>>>>> SESSION_IDLE_TIMEOUT�0

>>>>>
>>>>> #HTTPS stuff - secure proxy SSL header - do I need this?

>>>>> SECURE_PROXY_SSL_HEADER ('HTTP_X_FORWARDED_PROTO', 'https')

>>>>> #HTTPS stuff - secure cookies

>>>>> SESSION_COOKIE_SECURE True
>>>>> CSRF_COOKIE_SECURE True
>>>>>
>>>>> #HTTPS WSGI
>>>>> os.environ['wsgi.url_scheme'] 'https'

>>>>>>>>>>>> When I run Django/mod_wsgi/Apache on port 80 (same config as below, minus the mod_ssl stuff) or use the django development runserver 0.0.0.0:80, and disable the following settings in settings.py (#SESSION_COOKIE_SECURE True #CSRF_COOKIE_SECURE True) these browsers work correctly in the app.

>>>>>>>>>>>>
>>>>>>>>>>>> However, when running Django application running through mod_wsgi and HTTPS/port 443 in Apache, I see problems with both IE and Safari browsers. After login on Internet Explorer, page timeouts occur in various locations, reporting "This page can't be displayed". On Safari, the app won't get past the secondary Duo MFA authentication step, saying "Server unexpectedly dropped the connection." It is not a consistent behavior - seems to happen more frequently if I click quickly through links. Sometimes if I wait long enough to click, it might work momentarily, but then not again a moment later. This behavior does NOT happen using Chrome or Firefox browsers on any OS.
>>>>>>>>>>>>
>>>>>>>>>>>> Apache config:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> <IfModule mod_ssl.c>
>>>>>>>>>>>>
>>>>>>>>>>>> <VirtualHost *:443>
>>>>>>>>>>>>
>>>>>>>>>>>> ServerName **redacted**
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> #Django WSGI - Daemon
>>>>>>>>>>>>
>>>>>>>>>>>> WSGIScriptAlias / /var/www/transfergateway/myproject/apache/wsgi.py
>>>>>>>>>>>>
>>>>>>>>>>>> WSGIProcessGroup file-xfer
>>>>>>>>>>>>

>>>>>>>>>>>> WSGIDaemonProcess file-xfer user=mod_wsgi group=mod_wsgi processes=2 threads% python-path=/var/www/transfergateway

[Jason Garber]

Hi Jennifer,

Here is how we do it, and it works well.  Please adapt as needed.

In summary, we use nginx to terminate ssl, handle any redirects, serve static content, and proxy any remaining requests to apache.  Essentially apache is just a very robust application server at this point.

In Apache, each app is served on it's own port on 127.0.0.1.  This keeps the application traffic from getting mixed up.  Nginx contains the ability to serve http and https from the same server entry which is supremely convenient.

Keep in mind that nginx will proxy over HTTP/1.0 which means there is no Host header sent by default.  It is added in the nginx server.

Note that I have not used it with Django, but have used it with everything from Ruby and Phusion Passenger to PHP to Python and mod_wsgi.  It *just works*.

-----------------------------------------

NGINX:

Install from package repository

Here is a tweaked /etc/nginx/nginx.conf you can use:

https://github.com/appcove/acn-linux/blob/master/os-template/etc/nginx/nginx.conf

Note Line 22 needs updated

It includes from /etc/nginx/conf.d/*.conf

It includes from /etc/nginx/conf.server.d/*.conf

(you need to manually create that directory, or just remove that include line and put everything in conf.d)

-- Note: the following is included automatically by the main /etc/nginx/nginx.conf --

/etc/nginx/conf.server.d/www.example.com.conf

server

{

  listen 192.168.50.12:80;

  listen 192.168.50.12:443 ssl;

  server_name www.example.com;

  # Note, this is the /etc/nginx/ssl/ directory

  ssl_certificate     ssl/www.example.com.crt;

  ssl_certificate_key ssl/www.example.com.key;

  # Nginx uses HTTP 1.0 to proxy, so we need to manually add headers so the app can 

  # know if we were in http or https and who originally requested the content

  proxy_set_header Host $host;

  proxy_set_header X-Real-IP $remote_addr;

  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

  proxy_set_header Scheme $scheme;

  # Allow large uploads

  client_max_body_size 1000m;

  # Nginx serves static content, so it is important to forbid any

  # python files from being served

  location ~ \.(py|pyc|pyo|wsgi)$

  {

    return 403;

  }

  # Anything with an extension is served directly.  You may want to

  # remove this from your config.

  location ~ \.([a-zA-Z0-9])+$

  {

    root  /home/jason/ExampleProject/Web;

    add_header Cache-Control 'no-cache, no-store, max-age=0, must-revalidate';

    add_header Expires 'Thu, 01 Jan 1970 00:00:01 GMT';

  }

  # Everything else is proxied to Apache on 127.0.0.1 port 60301

  location /

  {

    add_header Cache-Control 'no-cache, no-store, max-age=0, must-revalidate';

    add_header Expires 'Thu, 01 Jan 1970 00:00:01 GMT';

    proxy_pass http://127.0.0.1:60301;

  }

}

The following apache config is included by the main httpd config file

# Notice we are telling apache to listen on 127.0.0.1 port 60301.

# For the sake of clairity, we are calling the WSGIDaemonProcess by the same name as the port.

# By hosting each app on it's own port, we eliminate any server-name issues between nginx and apache

WSGIDaemonProcess Port60301 processes=2 threads=2 python-path=/home/jason/ExampleProject/Python

Listen 127.0.0.1:60301

NameVirtualHost 127.0.0.1:60301

# www.example.com

<VirtualHost 127.0.0.1:60301>

  ServerName _default_

  DocumentRoot /home/jason/DevLevel.2/PBN/Web/Main

  AddDefaultCharset UTF-8

  RewriteEngine on

  RewriteOptions inherit

  # Forbid any python source files from being served.

  RewriteRule \.(py|pyc|pyo|wsgi)$  -  [F]

  WSGIScriptAlias / /home/jason/ExampleProject/Web/__init__.wsgi

  WSGIProcessGroup Port60301

  LogLevel info

  ErrorLog /home/jason/ExampleProject/apache-error.log

</VirtualHost>

These lines are also in the main apache config:

LoadModule wsgi_module modules/python33-mod_wsgi.so

WSGISocketPrefix run/wsgi

WSGIApplicationGroup %{GLOBAL}

------------------------

Thanks!

Jason

[Jennifer Mehl]

I gave this mod_wsgi-express a try, but it is spitting out an error about the —home option not being a valid option. What should I be using instead?

thanks,
Jennifer

>>> Setting DEBUG False seems to break my application - I get a “Bad Request 400” error back in my browser - so I will check in with the developer on that one.

>>>
>>> I’ve removed the extraneous environment variables and also the SSL proxy setting. I am only using mod_wsgi with Apache, so, as you say, it shouldn’t need that anyhow.
>>>
>>> I’ve done the test for the /wsgicheck and it does return a value of https. Thanks for helping me verify that functionality.
>>>
>>> So, this leaves me with looking at Apache as a culprit - or again, the Django code itself. It’s very odd how only the two browsers are showing issues and they are completely different issues…
>>>
>>> thanks,
>>> Jennifer
>>>
>>>
>>>
>>>> On Dec 16, 2014, at 4:41 PM, Graham Dumpleton <graham.d...@gmail.com> wrote:
>>>>
>>>> Hmmm, this looks really dangerous:
>>>>

>>>> DEBUG "FALSE"

>>>>
>>>> The DEBUG setting is meant to be a boolean value, not a string.
>>>>
>>>> Because you are setting it to a non empty string, it will be interpreted as True and so you have debug mode enabled.
>>>>
>>>> That is not good as sensitive information could be exposed back to users in error pages shown in the browser.
>>>>
>>>> Running in debug mode might cause other issues as well.
>>>>
>>>> Ensure you are setting it to:
>>>>

>>>> DEBUG False
>>>>
>>>> Also, setting:
>>>>
>>>> os.environ['HTTPS'] "on"
>>>> os.environ['wsgi.url_scheme'] 'https'

>>>>
>>>> will not do anything.
>>>>
>>>> The wsgi.url_scheme is an attribute which is passed down by mod_wsgi in the details for each request. A web framework will use the flag from the request details. The main thing it controls is merely the construction of absolute URLs when needing to be added to response headers or maybe response content in some cases.
>>>>
>>>> In other words, you do not need to set it and setting it in environment variables wouldn't do anything anyway.
>>>>
>>>> Next, setting:
>>>>
>>>> os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")
>>>>
>>>> is okay if you have just the one Django site, but be careful in using this if you are running more than one. Safer to use:
>>>>

>>>> os.environ["DJANGO_SETTINGS_MODULE"] "myproject.settings"

>>>>
>>>> More details in:
>>>>
>>>> http://blog.dscpl.com.au/2012/10/requests-running-in-wrong-django.html
>>>>
>>>> You also don't need:
>>>>

>>>> SECURE_PROXY_SSL_HEADER ('HTTP_X_FORWARDED_PROTO', 'https')

>>>>
>>>> if Apache is your front facing web server. You would only need this if you had a further front end proxy such as nginx in front of Apache and nginx had been configured to actually introduce these headers. That your Apache is accepting HTTPS requests would indicate that you don't have an nginx in front.
>>>>
>>>> Now as to determine whether wsgi.url_scheme is set properly, the easiest way is to take a copy of:
>>>>
>>>> def application(environ, start_response):

>>>> status '200 OK'
>>>> output str(environ.get('wsgi.url_scheme'))
>>>>

>>>> response_headers [('Content-type', 'text/plain'),

>>>>> os.environ['HTTPS'] "on"

>>>>>
>>>>> # This application object is used by any WSGI server configured to use this
>>>>> # file. This includes Django's development server, if the WSGI_APPLICATION
>>>>> # setting points here.
>>>>> from django.core.wsgi import get_wsgi_application

>>>>> application get_wsgi_application()

>>>>>
>>>>>
>>>>>
>>>>>
>>>>> and here is relevant stuff from my settings.py file:
>>>>>
>>>>> import os

>>>>> PROJECT_ROOT os.path.realpath(os.path.dirname(__file__))

>>>>>
>>>>>
>>>>> #turn off debug when going to production

>>>>> DEBUG "FALSE"
>>>>> TEMPLATE_DEBUG DEBUG

>>>>>
>>>>>
>>>>> # Python dotted path to the WSGI application used by Django's runserver.

>>>>> WSGI_APPLICATION 'myproject.wsgi.application'

>>>>>
>>>>>
>>>>> #session expire at browser close

>>>>> SESSION_EXPIRE_AT_BROWSER_CLOSE True
>>>>> SESSION_COOKIE_HTTPONLY=True
>>>>>
>>>>> #idle timeout
>>>>> SESSION_IDLE_TIMEOUT�0
>>>>>

>>>>> #HTTPS stuff - secure proxy SSL header - do I need this?

>>>>> SECURE_PROXY_SSL_HEADER ('HTTP_X_FORWARDED_PROTO', 'https')

>>>>> #HTTPS stuff - secure cookies

>>>>> SESSION_COOKIE_SECURE True
>>>>> CSRF_COOKIE_SECURE True
>>>>>
>>>>> #HTTPS WSGI
>>>>> os.environ['wsgi.url_scheme'] 'https'

>>>>>>>>>>>> When I run Django/mod_wsgi/Apache on port 80 (same config as below, minus the mod_ssl stuff) or use the django development runserver 0.0.0.0:80, and disable the following settings in settings.py (#SESSION_COOKIE_SECURE True #CSRF_COOKIE_SECURE True) these browsers work correctly in the app.

>>>>>>>>>>>>
>>>>>>>>>>>> However, when running Django application running through mod_wsgi and HTTPS/port 443 in Apache, I see problems with both IE and Safari browsers. After login on Internet Explorer, page timeouts occur in various locations, reporting "This page can't be displayed". On Safari, the app won't get past the secondary Duo MFA authentication step, saying "Server unexpectedly dropped the connection." It is not a consistent behavior - seems to happen more frequently if I click quickly through links. Sometimes if I wait long enough to click, it might work momentarily, but then not again a moment later. This behavior does NOT happen using Chrome or Firefox browsers on any OS.
>>>>>>>>>>>>
>>>>>>>>>>>> Apache config:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> <IfModule mod_ssl.c>
>>>>>>>>>>>>
>>>>>>>>>>>> <VirtualHost *:443>
>>>>>>>>>>>>
>>>>>>>>>>>> ServerName **redacted**
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> #Django WSGI - Daemon
>>>>>>>>>>>>
>>>>>>>>>>>> WSGIScriptAlias / /var/www/transfergateway/myproject/apache/wsgi.py
>>>>>>>>>>>>
>>>>>>>>>>>> WSGIProcessGroup file-xfer
>>>>>>>>>>>>

>>>>>>>>>>>> WSGIDaemonProcess file-xfer user=mod_wsgi group=mod_wsgi processes=2 threads% python-path=/var/www/transfergateway

[Graham Dumpleton]

Whoops. Should have been --working-directory instead of --home.

Graham

[Graham Dumpleton]

What I gave you was intended just to try and help you verify what the issue is, but be aware that --https-only is not working as intended in the typical case and is not forcing a HTTP request to be a HTTPS request. It only works if you also have used the --server-alias option.

I have a fix for the next version of mod_wsgi.

If there was some reason you really needed it for now, then add --server-alias with a dummy value.

--server-alias dummy.redacted

Graham

[Jennifer Mehl]

This was *VERY* helpful, thanks!

Looks like it was some weird behavior with Apache and SSL. I don’t think mod_wsgi was one of the troublemakers at all. :-)

Both of the browser’s bad behavior is now gone. I’ve changed to using Nginx as an SSL front-end proxy to a back-end Apache acting as an application web server for the Django application (still using mod_wsgi 4.4.1).

It took me a bit to figure out how to get the REMOTE_ADDR header to reflect the actual IP address instead of the proxy’s, and to pass that into the web application (used in several areas for rules and logging) but all is working well now.

Thanks to you and Graham for all of your help on getting me up and going!

—Jennifer

[Jason Garber]

Hi Jennifer,

https://github.com/gnif/mod_rpaf

mod_rpaf is an apache module which will consume the headers I have set in nginx.  Once installed it can be configured like this:

 LoadModule rpaf_module modules/mod_rpaf.so

 RPAFenable On

 RPAFsethostname On

 RPAFproxy_ips 127.0.0.1

 RPAFheader X-Forwarded-For

Long story short is that it will cause apache to behave normally with regards to logging and providing the IP to the application.  In the case of PHP and similar, you still need an alternate way to know if HTTP or HTTPS was used, which we use the HTTP_SCHEME variable for and just handle it in the application layer.

Thanks!

Jason

[Graham Dumpleton]

For wsgi.url_scheme, if you use mod_rewrite, mod_headers or mod_setenvif to detect that https is being indicate through a header set by the proxy, you can set the env parameter 'HTTPS' and mod_wsgi will detect that and automatically override wsgi.url_scheme to be 'https'.

Not sure how you are using Scheme header, but using mod_setenvif as one way, you can write: 

    SetEnvIf X-Scheme https HTTPS=1 

Do it this way and you don't have to fiddle it in your WSGi application.

Note that you should always consult wsgi.url_scheme and not look for HTTPS in environ for request, as mod_wsgi will purposely remove HTTPS after setting wsgi.url_scheme from it to try and stop people writing non portable WSGI applications.

Graham

[Jason Garber]

I was not aware of wsgi.url_scheme.  Thanks!

[Graham Dumpleton]

Is the official WSGI way of being notified that is HTTPS.

    https://www.python.org/dev/peps/pep-0333/#id19

Is what should be used by any WSGI framework when doing URL reconstruction.

    https://www.python.org/dev/peps/pep-0333/#url-reconstruction

Graham