In case of https, mod_wsgi-express setup-server command generates http.conf without Alias '/static'

[Tanuka Dutta]

Hello,

I have a CentOS 6.7 installation on a VM, and am running Django 1.8.8 over Apache 2.2.15 using mod_wsgi version 4.4.21 (that I had compiled from source using Python 2.7.8 in a virtualenv).

I've used mod_wsgi-express to generate the Apache scripts (it's extremely helpful!) and it works fine for http. I am able to connect to the website over http and view all the images on the site.

The commands used were:

(vishwaas_env)[syt_admin@VM1 www]$ mod_wsgi-express setup-server /home/syt_admin/projects/vishwaas/vishwaas_django/vishwaas_django/wsgi.py --host=0.0.0.0 --port=80 --user apache --group apache --server-root=/home/syt_admin/projects/vishwaas/www

(vishwaas_env)[syt_admin@VM1 www]$ sudo ./apachectl start

However, when I try to do the same for https (I followed instructions at https://gist.github.com/GrahamDumpleton/b79d336569054882679e, but left out the client authentication bit).

(vishwaas_env)[syt_admin@VM1 www-https]$ mod_wsgi-express setup-server ../vishwaas_django/vishwaas_django/wsgi.py --user apache --group apache --server-root=/home/syt_admin/projects/vishwaas/www-https --https-port 443 --port 80 --https-only --server-name www.xyz.in --ssl-certificate-file ssl-certs/server.crt --ssl-certificate-key-file ssl-certs/server.key

the http.conf file that is generated does not have the Alias 'static' directive. As I result, I can connect to the website but not view any images.

If I insert the missing lines (copied over from the http.conf generated earlier):

Alias '/static' '/home/syt_admin/projects/vishwaas/vishwaas_django/collected_static'

<Directory '/home/syt_admin/projects/vishwaas/vishwaas_django/collected_static'>
    Order allow,deny
    Allow from all
</Directory>

then I can see the static files on the website.

Is this a bug in mod_wsgi-express or am I doing something wrong?

Regards,

Tanuka

[Graham Dumpleton]

You would only get a Alias directive for /static if you had used the option:

    —url-alias /static /home/syt_admin/projects/vishwaas/vishwaas_django/collected_static

I can’t see that you have used that in what you ran.

The /static Alias would only be generated automatically if you were using the Django management command integration so you could run ‘python manage.py runmodwsgi', which you aren’t.

Graham

[Tanuka Dutta]

Thanks, Graham!

Adding the --url-alias option worked perfectly.

Also, I went over my old notes and realized that I had in fact executed "python manage.py runmodwsgi --setup-only" after having played around with the "mod_wsgi-express setup-server" command. So that explains the mystery of why it had worked in the past.

Is there a recommendation of which command to use?

Thanks for this great piece of infrastructure!!

Regards,

Tanuka

[Tanuka Dutta]

A follow-on question:

I was encountering issues with Chrome browser on Android 5.1.1 trying to connect to my website. The browser was flagging it as not secure. It turned out that my certificate chain was not properly installed on Apache. (My certificate was obtained from https://www.startssl.com/)

If I explicitly add this line in the httpd.conf where ever SSLCertificateFile is configured:

SSLCertificateChainFile /home/syt_admin/projects/vishwaas/www-https/startssl-certs/1_root_bundle.crt

(I am using Apache version 2.2.15)

then Android does not flag any issue and the browser shows the secure padlock icon.

But how do I provide this option via mod_wsgi-express setup-server ? I didn't see anything under mod_wsgi-express --help.

Regards,

Tanuka

--
You received this message because you are subscribed to a topic in the Google Groups "modwsgi" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/modwsgi/1Aad8DeGUjo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to modwsgi+u...@googlegroups.com.
To post to this group, send email to mod...@googlegroups.com.
Visit this group at https://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.

[Graham Dumpleton]

Is it replacing the SSLCertificateFile line or adding to it?

You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.

[Graham Dumpleton]

So according to:

http://httpd.apache.org/docs/2.2/mod/mod_ssl.html#sslcertificatechainfile

seems like I should provide a —ssl-certficate-chain-file option.

Because where it is placed is position sensitive, I don’t think it can simply be added to the end of the generated httpd.conf by using the —include-file option.

So will need to have in template:

SSLEngine On

SSLCertificateFile %(ssl_certificate_file)s

SSLCertificateKeyFile %(ssl_certificate_key_file)s

<IfDefine MOD_WSGI_VERIFY_CLIENT>

SSLCACertificateFile %(ssl_ca_certificate_file)s

SSLVerifyClient none

</IfDefine>

<IfDefine MOD_WSGI_CERTIFICATE_CHAIN>

SSLCertificateChainFile %(ssl_certificate_chain_file)s

</IfDefine>

and modify script to accepte —ssl-certficate-chain-file and enable inclusion of SSLCertificateChainFile.

I will make a change tomorrow and push it up to GitHub repo so you can verify it.

Graham

[Tanuka Dutta]

> Is it replacing the SSLCertificateFile line or adding to it?

Adding to it.

- Tanuka

[Tanuka Dutta]

>I will make a change tomorrow and push it up to GitHub repo so you can verify it.

OK. I guess that means that I will need to download the code and recompile mod_wsgi ? 

I am a relative newbie and find that a bit scary!

The last time I had done this, I had fetched mod_wsgi-4.4.21 using 

wget https://pypi.python.org/packages/source/m/mod_wsgi/mod_wsgi-4.4.21.tar.gz

and run "python setup.py install"  in my virtualenv (uses Python 2.7.8).

Would the process be the same this time around? Any gotchas that I need to be aware of?

- Tanuka

[Graham Dumpleton]

Quickly hacked it in while listening in to meeting. :-)

Can you try:

    pip install -U https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip

Then add the extra option:

    —ssl-certificate-chain-file some/path/file.crt

Tell me if that works and if does will add change note. Otherwise tell me how it fails with further clues for how generated httpd.conf should be set up.

Thanks.

Graham

[Graham Dumpleton]

On 22 Feb 2016, at 3:07 PM, Tanuka Dutta <tanuka...@gmail.com> wrote:

Thanks, Graham!

Adding the --url-alias option worked perfectly.

Also, I went over my old notes and realized that I had in fact executed "python manage.py runmodwsgi --setup-only" after having played around with the "mod_wsgi-express setup-server" command. So that explains the mystery of why it had worked in the past.

Is there a recommendation of which command to use?

As to this question, which you use really depends on which you think looks easier.

Technically the only reason you might not want to use 'python manage.py runmodwsgi’ is the fact that it does automatically set up the URL alias.

I have had someone who didn’t like that because they were using WSGI middleware called Whitenoise to handle static file serving and didn’t want Apache handling static files. They needed Whitenoise to handle static files as they were relying on special header Whitenoise would add which allow some fronted CDN to be used for caching. Right now when using 'python manage.py runmodwsgi’ there is no way to tell mod_wsgi-express to not handle static files and so allow it to fallback to Whitenoise. Using mod_wsgi-express, because you have to manually tell it when to handle static files you have full control to do it however you need.

Graham

[Graham Dumpleton]

On 22 Feb 2016, at 9:40 PM, Graham Dumpleton <graham.d...@gmail.com> wrote:

Quickly hacked it in while listening in to meeting. :-)

Can you try:

    pip install -U https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip

Then add the extra option:

    —ssl-certificate-chain-file some/path/file.crt

Tell me if that works and if does will add change note. Otherwise tell me how it fails with further clues for how generated httpd.conf should be set up

BTW. If you need to go back to the last official release, you would later run:

    pip install -U mod_wsgi==4.4.22

There is no need to download the tar file yourself and run ‘python setup.py install’. You can use ‘pip’ to install versions with it working out how to download files from the Python Package Index.

I would suggest you learn about how pip works if you don’t know too much about it as it is the standard way of installing Python packages.

Graham

[Tanuka Dutta]

Hi Graham,

It works fine!

The generated http.conf has sections like this that repeat for different VirtualHosts:

<VirtualHost _default_:443>

<Location />

Order deny,allow

Deny from all

<IfDefine MOD_WSGI_ALLOW_LOCALHOST>

Allow from localhost

</IfDefine>

</Location>

SSLEngine On

SSLCertificateFile /home/syt_admin/projects/vishwaas/www-https/startssl-certs/server.crt

SSLCertificateKeyFile /home/syt_admin/projects/vishwaas/www-https/startssl-certs/server.key

<IfDefine MOD_WSGI_VERIFY_CLIENT>

SSLCACertificateFile None

SSLVerifyClient none

</IfDefine>

<IfDefine MOD_WSGI_CERTIFICATE_CHAIN>

SSLCertificateChainFile /home/syt_admin/projects/vishwaas/www-https/startssl-certs/1_root_bundle.crt

</IfDefine>

</VirtualHost>

I am able to connect to the website using Chrome or IE on my Windows 8 laptop and also using Chrome on Android 5.1.1 - all browsers show the secure padlock icon.

I can also connect to the server using my Android app for the application.

Thanks so much! Will this become part of an official release that I can download later?

Regards,

Tanuka

[Graham Dumpleton]

Yes, it will be in the next release.

Right now the only change in that develop version I had you install relative to last released version is this specific change, so there should be no unexpected issues and should be safe to use for now.

Graham

[Tanuka Dutta]

Thank you for all the help!

- Tanuka

[Tanuka Dutta]

Hi Graham,

It has been a few days since I installed your modwsgi update with the knob for —ssl-certificate-chain-file some/path/file.crt.

Today, I needed to run "python manage.py collectstatic" and I encountered this error that I hadn't encountered earlier. I get the same error with any python manage.py xxx command.

I've looked at similar issues reported on StackOverflow but those answers did not work/apply in my case.

(vishwaas_env)[syt_admin@VM1 vishwaas_django]$ python manage.py collectstatic

Traceback (most recent call last):

  File "manage.py", line 12, in <module>

    execute_from_command_line(sys.argv)

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line

    utility.execute()

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute

    self.fetch_command(subcommand).run_from_argv(self.argv)

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/core/management/__init__.py", line 182, in fetch_command

    settings.INSTALLED_APPS

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__

    self._setup(name)

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup

    self._wrapped = Settings(settings_module)

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__

    mod = importlib.import_module(self.SETTINGS_MODULE)

  File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module

    __import__(name)

  File "/home/syt_admin/projects/vishwaas/vishwaas_django/vishwaas_django/settings/production.py", line 19, in <module>

    SECRET_KEY = get_secret('SECRET_KEY')

  File "/home/syt_admin/projects/vishwaas/vishwaas_django/vishwaas_django/settings/production.py", line 17, in get_secret

    raise ImproperlyConfigured(error_msg)

django.core.exceptions.ImproperlyConfigured: Set the SECRET_KEY environment variable

Note that launching Django using the Apache script generated by modwsgi works fine and I am able launch and connect to the website.

However, I do notice that my Django logging file is now written in the server-root directory instead of in the django project directory as it was earlier. They are different directories at the same level of nesting.

Here's what I had used to generate the httpd.conf:

(vishwaas_env)[syt_admin@VM1 www-https]$ mod_wsgi-express setup-server ../vishwaas_django/vishwaas_django/wsgi.py --user apache --group apache --server-root=/home/syt_admin/projects/vishwaas/www-https --https-port 443 --port 80 --server-name www.xyz.in --ssl-certificate-file startssl-certs/2_www.xyz.in.crt --ssl-certificate-key-file startssl-certs/server.key --ssl-certificate-chain-file startssl-certs/1_root_bundle.crt --url-alias /static/home/syt_admin/projects/vishwaas/vishwaas_django/collected_static

wsgi.py has the following contents:

import os, sys

sys.path.append('/home/syt_admin/projects/vishwaas/vishwaas_django')

os.environ['DJANGO_SETTINGS_MODULE'] = "vishwaas_django.settings.production"

manage.py has the following contents:

import os

import sys

if __name__ == "__main__":

    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "vishwaas_django.settings.production")

    from django.core.management import execute_from_command_line

    execute_from_command_line(sys.argv)

production.py has the following contents (relevant snippet):

from .base import *

import json

from django.core.exceptions import ImproperlyConfigured

# JSON-based secrets module located at same level as manage.py

with open(root("secrets.json")) as f:

    secrets = json.loads(f.read())

def get_secret(setting, secrets=secrets):

    """Get the secret variable or return explicit exception."""

    try:

        return secrets[setting]

    except KeyError:

        error_msg = "Set the {0} environment variable".format(setting)

        raise ImproperlyConfigured(error_msg)

SECRET_KEY = get_secret('SECRET_KEY')

where root is defined in settings/base.py as:

here = lambda *dirs: join(abspath(dirname(__file__)), *dirs)

BASE_DIR = here("..", "..")

root = lambda *dirs: join(abspath(BASE_DIR), *dirs)

Any idea what could have gone wrong?

Regards,

Tanuka

[Graham Dumpleton]

On 2 Mar 2016, at 7:56 PM, Tanuka Dutta <tanuka...@gmail.com> wrote:

Hi Graham,

The log file from Apache would always be in the server root directory unless you had used the --log-directory option to indicate a different directory that it should be written to.

So this suggests your secrets.json file doesn’t contain a SECRET_KEY. Is it definitely in there?

Have you tried logging the path for the secrets.json file calculated by root("secrets.json”)? Have you logged what the file contents were when read? Is the format of the file correct?

[Tanuka Dutta]

Hi Graham,

> The log file from Apache would always be in the server root directory unless you had used the --log-directory option to indicate a different directory that it should be written to

I don't mean the Apache error_log. It is the Django LOGGING feature that I'm using and the associated log file used to be written in the Django project directory before this. Now it is written inside my server-root. Is that expected ?

> So this suggests your secrets.json file doesn’t contain a SECRET_KEY. Is it definitely in there?

Yes, it is :-) I haven't touched the contents of that file.

> Have you tried logging the path for the secrets.json file calculated by root("secrets.json”)? Have you logged what the file contents were when read? Is the format of the file correct?

I added a print statement inside production.py to display this. The path computed by root("secrets.json") is correct.

However, one clue is that this line is printed twice. What would cause the settings file production.py to be loaded twice??

$python manage.py collectstatic

Path to secrets file /home/syt_admin/projects/vishwaas/vishwaas_django/secrets.json

Path to secrets file /home/syt_admin/projects/vishwaas/vishwaas_django/secrets.json

Traceback (most recent call last):

  File "manage.py", line 10, in <module>

    execute_from_command_line(sys.argv)

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line

    utility.execute()

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute

    self.fetch_command(subcommand).run_from_argv(self.argv)

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/core/management/__init__.py", line 182, in fetch_command

    settings.INSTALLED_APPS

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__

    self._setup(name)

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup

    self._wrapped = Settings(settings_module)

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__

    mod = importlib.import_module(self.SETTINGS_MODULE)

  File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module

    __import__(name)

  File "/home/syt_admin/projects/vishwaas/vishwaas_django/vishwaas_django/settings/production.py", line 22, in <module>

    SECRET_KEY = get_secret("SECRET_KEY")

  File "/home/syt_admin/projects/vishwaas/vishwaas_django/vishwaas_django/settings/production.py", line 20, in get_secret

    raise ImproperlyConfigured(error_msg)

django.core.exceptions.ImproperlyConfigured: Set the SECRET_KEY environment variable

Regards,

Tanuka

[Graham Dumpleton]

On 2 Mar 2016, at 9:04 PM, Tanuka Dutta <tanuka...@gmail.com> wrote:

Hi Graham,

> The log file from Apache would always be in the server root directory unless you had used the --log-directory option to indicate a different directory that it should be written to

I don't mean the Apache error_log. It is the Django LOGGING feature that I'm using and the associated log file used to be written in the Django project directory before this. Now it is written inside my server-root. Is that expected ?

That indicates that when setting Django logging you are likely using a relative path and not an absolute path.

In other words, the current working directory is not located where your project is. You should use the --working-directory option to specify the project directory as current working directory. Ensure the location of the log file uses an absolute path calculated relative to your code.

Better still, when running under mod_wsgi-express, don’t use a log file for Django logging. Set it up to write to the console. That way it ends up in the Apache error log. This is detailed in:

    http://blog.dscpl.com.au/2015/04/integrating-modwsgi-express-as-django.html

See section 'Logging of Python exceptions’.

> So this suggests your secrets.json file doesn’t contain a SECRET_KEY. Is it definitely in there?

Yes, it is :-) I haven't touched the contents of that file.

> Have you tried logging the path for the secrets.json file calculated by root("secrets.json”)? Have you logged what the file contents were when read? Is the format of the file correct?

I added a print statement inside production.py to display this. The path computed by root("secrets.json") is correct.

However, one clue is that this line is printed twice. What would cause the settings file production.py to be loaded twice??

$python manage.py collectstatic

Path to secrets file /home/syt_admin/projects/vishwaas/vishwaas_django/secrets.json

Path to secrets file /home/syt_admin/projects/vishwaas/vishwaas_django/secrets.json

This is due to the odd way that Django loads settings. I actually thought they had eliminated the double loading.

If is same issue, you can see an explanation of why it occurs in:

    http://blog.dscpl.com.au/2010/03/improved-wsgi-script-for-use-with.html

Ignore everything else in that blog post and is quite out of date now.

Traceback (most recent call last):

  File "manage.py", line 10, in <module>

    execute_from_command_line(sys.argv)

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line

    utility.execute()

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute

    self.fetch_command(subcommand).run_from_argv(self.argv)

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/core/management/__init__.py", line 182, in fetch_command

    settings.INSTALLED_APPS

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__

    self._setup(name)

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup

    self._wrapped = Settings(settings_module)

  File "/home/syt_admin/.virtualenvs/vishwaas_env/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__

    mod = importlib.import_module(self.SETTINGS_MODULE)

  File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module

    __import__(name)

  File "/home/syt_admin/projects/vishwaas/vishwaas_django/vishwaas_django/settings/production.py", line 22, in <module>

    SECRET_KEY = get_secret("SECRET_KEY")

  File "/home/syt_admin/projects/vishwaas/vishwaas_django/vishwaas_django/settings/production.py", line 20, in get_secret

    raise ImproperlyConfigured(error_msg)

django.core.exceptions.ImproperlyConfigured: Set the SECRET_KEY environment variable

I would suggest replicated the code that loads that file and looks up entries in a standalone small script and validate that it works as you expect. That is, take how you are hosting things and Django out of the picture.

Graham

[Tanuka Dutta]

Thanks for the tip, Graham.

I debugged the problem further by creating a standalone file that reads the secrets.json content and I am getting some strange results when I try to execute it using the python interpreter (2.7.8) in my virtualenv.

The secrets.json file is located correctly in the filesystem and its contents are read correctly, but then json.loads() seems to truncate each of the fields.

test_secrets.py

SECRETS_FILE =  root("secrets.json")

print "Path to secrets file %s" %SECRETS_FILE

with open(root("secrets.json")) as f:

    file_contents = f.read()

    print "File contents"

    print file_contents

    secrets = json.loads(file_contents)

print "secrets"

print secrets

Contents of secrets.json (with details obfuscated):

{

    "FILENAME": "secrets.json",

    "SECRET_KEY": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",

    "DATABASES_PASSWORD": "xxxxxxx"

}

The output of test_secrets.py is:

Path to secrets file /home/syt_admin/projects/vishwaas/vishwaas_django/secrets.json

File contents

{

    "FILENAME": "secrets.json",

    "SECRET_KEY": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",

    "DATABASES_PASSWORD": "xxxxxx"

}

secrets

{u'D\x00A\x00T\x00A\x00B\x00A\x00S\x00E\x00S\x00': u'J\x00a\x00J\x00i\x00', u'F\x00I\x00L\x00E\x00': u's\x00e\x00c\x00r\x00e\x00t\x00', u'S\x00E\x00C\x00R\x00E\x00': u'%\x000\x00t\x005\x00u\x00v\x00_\x00p\x00r\x00$\x00f\x00v\x004\x00g\x00k\x00p\x000\x00n\x00s\x00q\x00b\x005\x004\x00c\x00&\x00'}

As you can see, json.loads(file_contents) seems to be reading only partial contents of each key - DATABASES, FILE, SECRE, and also partial values for each.

What could be going wrong here? I thought the JSON size limit for a key is 255 characters...

I also realized that this is the first time I have tried to execute any "python manage.py xxx" after implementing the secrets.json method of reading values into my settings file. It is entirely coincidental that I upgraded mod-wsgi around the same time!

But why does "sudo ./apachectl start" work fine with the same settings file? The output that I see in my Apache error_log (printed from my settings file) has the entire contents of secrets.json correctly:

[Thu Mar 03 11:28:28 2016] [error] {u'DATABASES_PASSWORD': u'xxxxx', u'SECRET_KEY': u'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx', u'FILENAME': u'secrets.json'}

What is the difference?

I will need to run various python manage.py commands on the Linux server so I do need to fix this!

Regards,

Tanuka

[Tanuka Dutta]

Additional info: the secrets.json file is encoded as follows:

$file -i secrets.json

secrets.json: text/plain; charset=us-ascii

Regards,

Tanuka

[Tanuka Dutta]

I've found a solution but I don't understand why it works.

I stumbled across this blog

http://owtrsnc.blogspot.in/2012/04/handling-control-characters-escaping.html

that mentioned using encoding="utf8" instead of the default "utf-8".

I tried that in my code and got the desired output.

test_secrets.py

with open(root("secrets.json")) as f:

    file_contents = f.read()

    print "File contents"

    print file_contents

    secrets = json.loads(file_contents, encoding="utf8")

print "secrets"

print secrets

Output

File contents

{

    "FILENAME": "secrets.json",

    "SECRET_KEY": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",

    "DATABASES_PASSWORD": "xxxxxxx!"

}

secrets

{u'DATABASES_PASSWORD': u'xxxxxx', u'SECRET_KEY': u'xxxxxxxxxxxxxxxxxxxxxxxxx', u'FILENAME': u'secrets.json'}

Using the encoding="utf8" parameter allows me to load my settings when I execute

$python manage.py collectstatic

and it also works for 

$sudo ./apachectl start

What is the difference between encoding="utf8" vs "utf-8"? I haven't been able to find any reference to "utf8" in the Python JSON documentation.

Regards,

Tanuka

[Tanuka Dutta]

Hi Graham,

I see that the latest official release on https://pypi.python.org is still 4.4.22.

Any timeframe set for the next official release that would incorporate the extra option —ssl-certificate-chain-file some/path/file.crt ?

Meanwhile, I'm continuing with the developer version that you gave me.

Regards,

Tanuka

[Graham Dumpleton]

The latest in PyPi should be 4.5.2. Make sure you aren't following link by version, see latest.

Graham

> On 26 May 2016, at 00:29, Tanuka Dutta <tanuka...@gmail.com> wrote:
>
> Hi Graham,
>
> I see that the latest official release on https://pypi.python.org is still
> 4.4.22.
>
> Any timeframe set for the next official release that would incorporate the

> extra option =E2=80=94ssl-certificate-chain-file some/path/file.crt ?

[Tanuka Dutta]

OK, thanks. I was looking under the 4.4.x series.

I assume 4.5.2 incorporates this change?

Regards,

Tanuka

[Graham Dumpleton]

The —ssl-certificate-chain-file option was included in 4.4.23.

http://modwsgi.readthedocs.io/en/develop/release-notes/version-4.4.23.html

Graham

[Tanuka Dutta]

Thanks, Graham.

I'm encountering a different problem with Apache 2.4 and mod_wsgi 4.4.22. I am able to upload media files but unable to view them through the browser.

This is running on a RHEL 7.2 server with Django 1.8.8.

I used this command to generate the httpd.conf:

$ python manage.py runmodwsgi --setup-only  --port=80 --user=apache --group=apache --server-root=/home/syt_admin/projects/vishwaas/www --url-alias /media /home/syt_admin/projects/vishwaas/www/media

If I search for the string 'media' in the generated httpd.conf, I see these lines:

Alias '/media' '/home/syt_admin/projects/vishwaas/www/media'

<Directory '/home/syt_admin/projects/vishwaas/www'>

<Files 'media'>

    Order allow,deny

    Allow from all

</Files>

</Directory>

My Django settings file has the corresponding definitions:

MEDIA_ROOT = '/home/syt_admin/projects/vishwaas/www/media/'

MEDIA_URL = '/media/'

The files get uploaded to the correct location, by user 'apache'. But when I click on the link to the file in the browser, I get the error message: You don't have permission to access /media/<filename> on this server.

Also, the Apache error_log contains this error:

[Fri May 27 06:45:05.181364 2016] [access_compat:error] [pid 24953:tid 140221960361728] [client <ip address>:<port>] AH01797: client denied by server configuration: /home/syt_admin/projects/vishwaas/www/media/<filename>, referer: http://<ip address>/admin/schools/school/1/

I know that the Alias directives for Apache 2.4 are supposed to be different ie "Require all granted" but I don't see that in the generated httpd.conf file. 

Am I missing some option when invoking runmodwsgi??

Regards,

Tanuka

[Graham Dumpleton]

On 27 May 2016, at 3:57 AM, Tanuka Dutta <tanuka...@gmail.com> wrote:

Thanks, Graham.

I'm encountering a different problem with Apache 2.4 and mod_wsgi 4.4.22. I am able to upload media files but unable to view them through the browser.

This is running on a RHEL 7.2 server with Django 1.8.8.

I used this command to generate the httpd.conf:

$ python manage.py runmodwsgi --setup-only  --port=80 --user=apache --group=apache --server-root=/home/syt_admin/projects/vishwaas/www --url-alias /media /home/syt_admin/projects/vishwaas/www/media

If I search for the string 'media' in the generated httpd.conf, I see these lines:

Alias '/media' '/home/syt_admin/projects/vishwaas/www/media'

<Directory '/home/syt_admin/projects/vishwaas/www'>

<Files 'media'>

    Order allow,deny

    Allow from all

</Files>

</Directory>

Instead of this try:

<Directory '/home/syt_admin/projects/vishwaas/www/media'>

    Order allow,deny
    Allow from all

</Directory>

Am not sure that Files can be used on a directory, only files.

Graham

[Tanuka Dutta]

Yes, this works. I hadn't spotted that difference. Thank you!

I also re-ran runmodwsgi and the httpd.conf now contains the correct directives without the <Files>.

I don't know what I might have done wrong earlier...

How is this working with Apache 2.4, shouldn't the syntax now be "Require all granted"?

Regards,

Tanuka

[Graham Dumpleton]

On 28 May 2016, at 4:50 AM, Tanuka Dutta <tanuka...@gmail.com> wrote:

Yes, this works. I hadn't spotted that difference. Thank you!

I also re-ran runmodwsgi and the httpd.conf now contains the correct directives without the <Files>.

I don't know what I might have done wrong earlier…

Whoops, I missed that you were using mod_wsgi-express as mentioned in subject. Thus this is sort of my fault. I myself have been caught by this indirectly just last week as well and didn’t tweak was same issue. I never looked at the generated configuration in my case to realise Files was used how it was as I just fixed the underlying problem.

Anyway, the problem was caused by the fact that the file system directory you were referencing using —url-alias didn’t exist at the time mod_wsgi-express was run to generate the setup.

When —url-alias is used I need to do a check for trailing slash on the path to try and fix up a potential mistake by a user where usage of slashes on URL and target directory weren’t balanced. If I don’t then Apache doesn’t process things properly. Part of that check looks at the target path and if it is a directory does things one way and if not, does it another way.

Because the check is to see whether it is a directory, it causes a problem if the directory doesn’t exist at that point. I will have to go back and look at the code again and perhaps change the logic around so that rather than fallback to assuming the target path is a file, which is less likely case, assume it will be a directory unless the target path exists and is a file. That way it will work if the directory doesn’t exist yet. Does mean it will not then work properly in case where target path is a file but doesn’t at that point, but that is less likely scenario and I have to fallback one way or the other and can’t avoid one case loosing out where stuff doesn’t exist yet.

How is this working with Apache 2.4, shouldn't the syntax now be "Require all granted”?

Apache 2.4 has a mod_access_compat module which provides older directives in Apache 2.4. I rely on that so that I don’t have to have switch log on all the places where these directives are used.

Graham

[Tanuka Dutta]

Thanks, that explains exactly what happened. I wanted user Apache to create the media directory so I had not manually created it the first time. The file upload operation created the media directory, but by then my http d.conf had already been generated (incorrectly).

When I re- ran runmodwsgi the second time, of course the directory was present, so httpd.conf was correctly generated.

Good to understand what happens under the hood!

- Tanuka

Sent from my iPhone

[Tanuka Dutta]

Hi Graham,

Reviving an old thread...I think that the —url-alias option in runmodwsgi isn't always working as expected (or I'm doing something wrong). 

This is with Apache 2.4, Django 1.8.8 and mod_wsgi 4.4.23 running on a CentOS 6.7 server. 

Recently, I was able to upload media files but unable to view them through the browser.

Troubleshooting made me realize that some time back, when I had to update my ssl certificates, I had executed runmodwsgi but forgot to the include the option

—url-alias /media /home/syt_admin/projects/vishwaas/www-https/media

So naturally, the httpd.conf file did not contain the directive for /media

I tried to fix it by regenerating the httpd.conf file as follows:

python manage.py runmodwsgi --setup-only  --https-port=443 --port=80 --server-name=www.xyz.in --user=apache --group=apache --server-root=/home/syt_admin/projects/vishwaas/www-https --ssl-certificate-file=/home/syt_admin/projects/vishwaas/www-https/startssl-certs/2_www.xyz.in.crt --ssl-certificate-key-file=/home/syt_admin/projects/vishwaas/www-https/startssl-certs/server.key --ssl-certificate-chain-file=/home/syt_admin/projects/vishwaas/www-https/startssl-certs/1_root_bundle.crt —url-alias /media /home/syt_admin/projects/vishwaas/www-https/media

However, the generated httpd.conf file still does not contain the /media directive. 

Finally I had to add these lines by hand

Alias '/media' '/home/syt_admin/projects/vishwaas/www-https/media'

<Directory '/home/syt_admin/projects/vishwaas/www-https/media'>

    Order allow,deny
    Allow from all
</Directory>

This works, and I can view the files by clicking the link on the browser, but I'm still puzzled by the behaviour.

The directory was existing when I ran the command and it contained files inside it.

Any thoughts?

Regards,

Tanuka

[Graham Dumpleton]

I know of no issue where was been getting left out.

One thing to be careful of is if you run this and the directory the URL alias refers to doesn't exist, it will assume that you are actually trying to alias to a file.

For example:

    mod_wsgi-express start-server --url-alias /media /tmp/xxx

generates:

Alias '/media' '/tmp/xxx'

<Directory '/tmp'>

<Files 'xxx'>

<IfVersion < 2.4>

    Order allow,deny

    Allow from all

</IfVersion>

<IfVersion >= 2.4>

    Require all granted

</IfVersion>

</Files>

</Directory>

If you subsequently create the directory, it will not behave as you expect and any files in the directory probably will not be found due to Files directive not matching since it actually a directory.

If the directory did exist to begin with, you would get:

Alias '/media' '/tmp/xxx'

<Directory '/tmp/xxx'>

    AllowOverride None

<IfVersion < 2.4>

    Order allow,deny

    Allow from all

</IfVersion>

<IfVersion >= 2.4>

    Require all granted

</IfVersion>

</Directory>

I can't remember if the behaviour of this may have changed from a very long time ago, with it once upon a time assuming it was a directory unless the target existed and was a file. I know I have been caught by this as well and contemplated changing it to assume a directory if target doesn't exist. I need to go back and look at how I might have done this in the past.

I can only suggest you run the same setup command against an empty server root directory, where the media directory has been pre-created and see what it creates in the config in that case.

Also, what is the command that gets recorded in the generated 'apachectl' file. There is a possibility I have stuffed up something in the Django management command wrapper and it is overriding the url alias list instead of adding to it.

Graham