Problems getting mod_wsgi 3.4 and chroot working

[Jennifer Mehl]

Hello,

I'm new to mod_wsgi (and I am not a programmer, so please bear with me).  I have a Django/Python application that was written by someone else that I have deployed using mod_wsgi 3.4 on Ubuntu 14.04LTS with Apache 2.4.7, using the following in my apache config:

#Django WSGI

WSGIScriptAlias / /var/www/transfergateway/myproject/wsgi.py

WSGIPythonPath /var/www/transfergateway

<VirtualHost *:443>

    ServerName *redacted* 

        <Directory /var/www/transfergateway>

<Files wsgi.py>

Order deny,allow

Allow from all

</Files>

</Directory>

The application is running fine using mod_wsgi and apache.

However, I want to make this application more secure, so I would like to have mod_wsgi run in a chroot jail.  I created a chroot jail using the instructions here: https://help.ubuntu.com/community/BasicChroot and copied the application into /var/chroot/var/www/transfergateway .  Then, in the chroot, I installed python2.7 and used pip install to install the python packages used in the project (as far as I can tell).

I have created a new Apache config:

#test chroot jail for Django WSGI

WSGISocketPrefix /var/run/wsgi

<VirtualHost *:443>

    ServerName *redacted* 

WSGIScriptAlias / /var/chroot/var/www/transfergateway/myproject/wsgi.py

WSGIProcessGroup chroot 

WSGIDaemonProcess chroot user=daemon group=daemon processes=2 threads=25 chroot=/var/chroot

        <Directory /var/chroot/var/www/transfergateway/myproject/>

<Files wsgi.py>

Order deny,allow

Allow from all

</Files>

</Directory>

But I am getting the following errors in the Apache error log upon startup:

Target WSGI script '/var/www/transfergateway/myproject/wsgi.py' cannot be loaded as Python module.

Exception occurred processing WSGI script '/var/www/transfergateway/myproject/wsgi.py'.

Traceback (most recent call last):

File "/var/www/transfergateway/myproject/wsgi.py", line 29, in <module>

from django.core.wsgi import get_wsgi_application

File "/usr/local/lib/python2.7/dist-packages/django/core/wsgi.py", line 1, in <module>

 from django.core.handlers.wsgi import WSGIHandler

 File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/wsgi.py", line 9, in <module>

from django import http

File "/usr/local/lib/python2.7/dist-packages/django/http/__init__.py", line 1, in <module>

from django.http.cookie import SimpleCookie, parse_cookie

File "/usr/local/lib/python2.7/dist-packages/django/http/cookie.py", line 3, in <module>

from django.utils.encoding import force_str

File "/usr/local/lib/python2.7/dist-packages/django/utils/encoding.py", line 4, in <module>

import datetime

ImportError: No module named datetime

I appreciate any advice on what I am doing wrong here.  Do I have my chroot set up properly?  Do I need to do anything with permissions on /var/chroot or /var/chroot/var/www/transfergateway?  Do I need to run mod_wsgi as a different user than daemon?

thanks in advance for any help!

--Jennifer

[Graham Dumpleton]

Wow. Surprised you got that far. The chroot feature of mod_wsgi pretty well has no documentation. At best there might be some comments about it buried in the release notes somewhere.

Before we try and sort out the issue, I might say that the better way of trying to isolate an application these days would be to use Docker. I realise this means learning a bit about how to install and manage Docker, but as far as running Apache/mod_wsgi under Docker, the experience is much much better as I provide a prebuilt Docker image for doing it.

This Docker image is something I don't think I have mentioned here on the mod_wsgi mailing list as it has only been out there for a week or so.

I have recently started blogging about it and have two posts up about it:

Hosting Python WSGI applications using Docker.

Deferred build actions for Docker images.

The actual Docker Hub entry is:

https://registry.hub.docker.com/u/grahamdumpleton/mod-wsgi-docker/

Anyway, for chroot, can you confirm a few things.

First is whether the Python version outside of the chroot is the default operating system Python installation for 2.7 and that the mod_wsgi is also the operating system binary package also.

Double check what version of Python mod_wsgi is installed for. I would imagine it should be Python 2.7, but want to make sure isn't 2.6.

  http://code.google.com/p/modwsgi/wiki/CheckingYourInstallation#Python_Installation_In_Use

Now inside of the chroot, did you also use the default operating system Python installation for 2.7.

Inside of the chroot, run the 'python' command line and see if datetime can imported.

$ python

Python 2.7.2 (default, Oct 11 2012, 20:14:37)

[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin

Type "help", "copyright", "credits" or "license" for more information.

>>> import datetime

>>> datetime.__file__

'/Users/graham/Python/docker/lib/python2.7/lib-dynload/datetime.so'

Graham

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To post to this group, send email to mod...@googlegroups.com.
Visit this group at http://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.

[Graham Dumpleton]

Were you by chance using a Python virtual environment?

Apparently if your virtual environment is created of system Python and then you upgrade the system Python, then datetime can stop working.

http://askubuntu.com/questions/509283/python-no-module-named-datetime/542673#542673

Graham

[Jennifer Mehl]

You know, I'm not totally sure about the virtualenv... I am new to Python. We have had a bit of history with two different developers working on this code (and I am not in communication with the first developer).  Kind of a difficult spot I'm in, with limited info.  The first developer had started out using a virtualenv but I believe switched to using pip to install packages system-wide.   I am trying to normalize the package installation so they are all installed the same way and get a list of what packages are in use.

(I know this may be out of scope for the group, but if someone can tell me how to determine if a virtualenv is in use, and if so, how to deactivate it and move over to only using system-wide packages, that would be enormously helpful.)

Is the datetime error a separate error from the "Target WSGI" error, or causing it?

In the meantime, I will check the Python versions but I believe it is 2.7.6 on the system and chroot - not sure what the mod_wsgi is compiled against, will check.  It's the one that came through Ubuntu, so worst case, I could compile my own mod_wsgi if I need to.

Thanks for the help.

--Jennifer

[Graham Dumpleton]

On 12/12/2014, at 8:04 AM, Jennifer Mehl <jennif...@gmail.com> wrote:

You know, I'm not totally sure about the virtualenv... I am new to Python. We have had a bit of history with two different developers working on this code (and I am not in communication with the first developer).  Kind of a difficult spot I'm in, with limited info.  The first developer had started out using a virtualenv but I believe switched to using pip to install packages system-wide.   I am trying to normalize the package installation so they are all installed the same way and get a list of what packages are in use.

(I know this may be out of scope for the group, but if someone can tell me how to determine if a virtualenv is in use, and if so, how to deactivate it and move over to only using system-wide packages, that would be enormously helpful.)

As there is no evidence in the Apache configuration file for the use of a virtual environment, would need to look in the wsgi.py to see if set up there.

I am not sure though whether setting up in in the wsgi.py can trigger that datetime issue as described.

Important thing at this point is to check that the system wide Python and chroot Python are exactly the same because although your application will use the Python run time from inside the chroot, the actual Python shared library is going to be pulled from the system Python so any difference between the two Python installations version wise can cause problems.

Is the datetime error a separate error from the "Target WSGI" error, or causing it?

The datetime module not being able to be loaded is triggering the 'Target WSGI' error.

[Jennifer Mehl]

On Wednesday, December 10, 2014 6:13:56 PM UTC-8, Graham Dumpleton wrote:

Wow. Surprised you got that far. The chroot feature of mod_wsgi pretty well has no documentation. At best there might be some comments about it buried in the release notes somewhere.

Before we try and sort out the issue, I might say that the better way of trying to isolate an application these days would be to use Docker. I realise this means learning a bit about how to install and manage Docker, but as far as running Apache/mod_wsgi under Docker, the experience is much much better as I provide a prebuilt Docker image for doing it.

This Docker image is something I don't think I have mentioned here on the mod_wsgi mailing list as it has only been out there for a week or so.

I have recently started blogging about it and have two posts up about it:

Hosting Python WSGI applications using Docker.

Deferred build actions for Docker images.

The actual Docker Hub entry is:

https://registry.hub.docker.com/u/grahamdumpleton/mod-wsgi-docker/

Thanks for this info - I may look into this if the chroot proves too difficult.

 

Anyway, for chroot, can you confirm a few things.

First is whether the Python version outside of the chroot is the default operating system Python installation for 2.7 and that the mod_wsgi is also the operating system binary package also.

Double check what version of Python mod_wsgi is installed for. I would imagine it should be Python 2.7, but want to make sure isn't 2.6.

 

  http://code.google.com/p/modwsgi/wiki/CheckingYourInstallation#Python_Installation_In_Use

It shows:

sys.version = '2.7.6 (default, Mar 22 2014, 23:03:41) \n[GCC 4.8.2]'
sys.prefix = '/usr'

 

Now inside of the chroot, did you also use the default operating system Python installation for 2.7.

Inside of the chroot, run the 'python' command line and see if datetime can imported.

$ python

Python 2.7.2 (default, Oct 11 2012, 20:14:37)

[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin

Type "help", "copyright", "credits" or "license" for more information.

>>> import datetime

>>> datetime.__file__

'/Users/graham/Python/docker/lib/python2.7/lib-dynload/datetime.so'

Here's my result from inside the chroot:

Python 2.7.6 (default, Mar 22 2014, 22:59:38) 

[GCC 4.8.2] on linux2

Type "help", "copyright", "credits" or "license" for more information.

>>> import datetime

>>> datetime.__file__

'/usr/lib/python2.7/lib-dynload/datetime.i386-linux-gnu.so'

Do you think I should I try to compile a different version of mod_wsgi from source?

--Jennifer

[Jennifer Mehl]

I believe my Python versions are all in sync (2.7.6).

I discovered that my first problem was that my chroot was not completely set up.  I had neglected to mount the chroot fs in my guest.  I did that and have verified that the chroot is working as expected.  It is located in /var/chroot and the Django application has been copied to /var/chroot/var/www/transfergateway . (I have not changed any permissions on any of the chroot files).

Now that I have resolved the chroot problem, I am having a new error when trying to start Apache with running mod_wsgi in the chroot:

Updated apache.conf:

WSGISocketPrefix /var/run/wsgi

<VirtualHost *:443>

    ServerName **redacted***

WSGIScriptAlias / /var/chroot/var/www/transfergateway/myproject/wsgi.py

WSGIProcessGroup chroot

WSGIDaemonProcess chroot user=daemon group=daemon processes=2 threads=25 chroot=/var/chroot python-path=/var/chroot/var/www/transfergateway

        <Directory /var/chroot/var/www/transfergateway/myproject/>

                <Files wsgi.py>

                Order deny,allow

                Allow from all

                </Files>

        </Directory>

</VirtualHost>

WSGI File in chroot: /var/chroot/var/www/transfergateway/myproject/wsgi.py:

"""

WSGI config for myproject project.

This module contains the WSGI application used by Django's development server

and any production WSGI deployments. It should expose a module-level variable

named ``application``. Django's ``runserver`` and ``runfcgi`` commands discover

this application via the ``WSGI_APPLICATION`` setting.

Usually you will have the standard Django WSGI application here, but it also

might make sense to replace the whole Django WSGI application with a custom one

that later delegates to the Django one. For example, you could introduce WSGI

middleware here, or combine a Django application with an application of another

framework.

"""

import os

import sys

path='/var/www/transfergateway/myproject'

#if path not in sys.path:

#sys.path.append(path)

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")

# This application object is used by any WSGI server configured to use this

# file. This includes Django's development server, if the WSGI_APPLICATION

# setting points here.

from django.core.wsgi import get_wsgi_application

application = get_wsgi_application()

# Apply WSGI middleware here.

# from helloworld.wsgi import HelloWorldApplication

# application = HelloWorldApplication(application)

Apache error log:

mod_wsgi (pid=16343): Target WSGI script '/var/www/transfergateway/myproject/wsgi.py' cannot be loaded as Python module.

mod_wsgi (pid=16343): Exception occurred processing WSGI script '/var/www/transfergateway/myproject/wsgi.py'.

Traceback (most recent call last):

File "/var/www/transfergateway/myproject/wsgi.py", line 31, in <module> application = get_wsgi_application()

File "/usr/local/lib/python2.7/dist-packages/django/core/wsgi.py", line 14, in get_wsgi_application django.setup()

File "/usr/local/lib/python2.7/dist-packages/django/__init__.py", line 20, in setup configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)

File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 46, in __getattr__self._setup(name)

File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 42, in _setupself._wrapped = Settings(settings_module)

File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 98, in __init__% (self.SETTINGS_MODULE, e)

ImportError: Could not import settings 'myproject.settings' (Is it on sys.path? Is there an import error in the settings file?): No module named myproject.settings

I believe my issue now is relating to the PythonPath directive and access to the project/Python my chroot, but am unsure how to resolve.

I'm kind of stuck-- so I appreciate the help!

--Jennifer

[Joonas Lehtolahti]

On Fri, 12 Dec 2014 22:55:42 +0200, Jennifer Mehl
<jennif...@gmail.com> wrote:

...

> WSGIDaemonProcess chroot user=daemon group=daemon processes=2 threads=25
> chroot=/var/chroot python-path=/var/chroot/var/www/transfergateway

...

> I believe my issue now is relating to the PythonPath directive and access
> to the project/Python my chroot, but am unsure how to resolve.
>
> I'm kind of stuck-- so I appreciate the help!
>
> --Jennifer
>

Just a thought from an outsider: since the Python interpreter is running
inside chroot, perhaps the python-path option needs to point to a path
local to the root, that is, it shouldn't contain /var/chroot.

Cheers,

Joonas

[Graham Dumpleton]

Yes, that is correct.

General Apache configuration should use /var/chroot as well as WSGIScriptAlias.

But anything that would be applied when handling a request shouldn't have /var/chroot as it is being used by the application inside of the chroot.

So I would agree that should probably be:

WSGIDaemonProcess chroot user=daemon group=daemon processes=2 threads=25 \

chroot=/var/chroot python-path=/var/www/transfergateway

Thanks Joonas, you beat me to it. :-)

Graham

[Graham Dumpleton]

BTW, for absolute paths in your Python code such as Django settings module, it is important that if want to run application both in and out of the chroot, that you don't use absolute paths and instead calculate them.

For example in Django settings, construct them relative to BASEDIR calculated from __file__.

BASE_DIR = os.path.dirname(os.path.dirname(__file__))

[Jennifer Mehl]

Thank you Joonas and Graham - you guys are lifesavers!  That really helped.

This moves me to the next error:

mod_wsgi (pid=17140): Target WSGI script '/var/www/transfergateway/myproject/wsgi.py' cannot be loaded as Python module.

mod_wsgi (pid=17140): Exception occurred processing WSGI script '/var/www/transfergateway/myproject/wsgi.py'.

Traceback (most recent call last):

File "/var/www/transfergateway/myproject/wsgi.py", line 30, in <module> application = get_wsgi_application()

File "/usr/local/lib/python2.7/dist-packages/django/core/wsgi.py", line 14, in get_wsgi_application django.setup()

File "/usr/local/lib/python2.7/dist-packages/django/__init__.py", line 21, in setup apps.populate(settings.INSTALLED_APPS)

File "/usr/local/lib/python2.7/dist-packages/django/apps/registry.py", line 89, in populate

 "duplicates: %s" % app_config.label) ImproperlyConfigured: Application labels aren't unique, duplicates: sessions

I am not the developer of this application, so I'm not sure where to look to fix this.  I did install all of the dependent django and python packages that I know of, inside the chroot.

--Jennifer

[Graham Dumpleton]

What do you have in INSTALLED_APPS of the Django settings file?

Graham

[Jennifer Mehl]

INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
# Uncomment the next line to enable the admin:
'django.contrib.admin',
# Uncomment the next line to enable admin documentation:
'django.contrib.admindocs',
'myproject.duo_app',
'myproject.myapp',
'sessions',
)

How can I figure out which of the 2 sessions apps listed, is in use? And if they are both in use, I presume I need to rename one of them in the code?

I realize this out of scope for the mod_wsgi group, but do appreciate the help. :-)

—Jennifer

> You received this message because you are subscribed to a topic in the Google Groups "modwsgi" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/modwsgi/sOrtcm7JV50/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to modwsgi+u...@googlegroups.com.

[Graham Dumpleton]

I am not a Django person, but strikes me that:

django.contrib.sessions
sessions

would conflict as application name likely taken from name of last sub module in path.

Graham