How to run Apache and mod_wsgi using system python not some virtual python

[OwlHoot]

I have used mod_wsgi-express to whump up an Apache config. But I found a problem whereby when run by user "apache" it can't find a python module which I know is present when running under "root" :

# whoami
root

# python -c "import django;"
#

# su apache

$ python -c "import django;"

  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'django'

I recall reading that mod_wsgi and mod_wsgi-express is installed with python running in some virtual environment. But as I am running this in a Docker container, I am really not interested in and know little about outdated old stuff like venv, virtualenv, and suchlike (and I wish there was a mod_wsgi-express option to use the system-wide python, and not bother with spinning a virtual cocoon round the python it uses, as doing this simply complicates things when using Docker! )

So my question is how can I "devirtualise" this python that runs under user "apache", so it runs the system-wide python3 and its libraries, and thus pick up the django module among others, the same as "root"?

Thanks in anticipation

Regards

John R Ramsden

P.S. The annoying thing is I had the perishing thing working on Friday, and carefully preserved my changes, or thought I had. But upon rebuilding the Docker container, something is not quite the same as before and the Apache error log has reverted to containing the "Not found 'django'" error as above. I'd be happy to set the access right to every file in the container to 777 if that is what it takes, as this app will be running inside an intranet. But I have a feeling that won't fix the problem, and probably isn't entirely sound from a security standpoint.

P.S. As a service to this mailing list, which I hope others may find useful, I append a python script I wrote to strip out all the extra "ifs and buts" cruft from the generated httpd.conf file, so one can see more clearly what is going on. (IMHO it would be a good idea to have a mod_wsgi-express option that generates the httpd.conf file without them to start with.) The script may not win any prizes for elegance, but it did the job for me. (But use it with care, as some version numbers are hard-coded! )

#!/usr/bin/python

import re

env_vars = dict()

#  Fetch '-D' tokens from httpd command arguments
#
#    HTTPD_ARGS="-f /application/apache/httpd.conf -E /application/apache/startup.log -DMOD_WSGI_VIRTUAL_HOST -DMOD_WSGI_WITH_HTTPS .."
#
funit = open('apachectl', 'r')

lines = funit.read().splitlines()

httpd_args = 'HTTPD_ARGS='

for line in lines:

    if len(httpd_args) < len(line) and httpd_args == line[0:len(httpd_args)]:

        matches = re.finditer(' -D[A-Z_]+', line)

        for match in matches:
            env_vars[match.group()[3:]] = 1

        break

funit.close()


#  Output httpd.conf file with Ifs and Buts removed
#
n_blank_line = 0

old_version = False

skip_level = 0

skip_flags = []

defined_no  = '<IfDefine !'
defined_yes = '<IfDefine '

funit = open('httpd.conf', 'r')

lines = funit.read().splitlines()

for line in lines:

    skip = None

    if '<IfVersion >= 2.2.15>' == line or '<IfVersion >= 2.4>' == line:
        continue

    elif '<IfVersion < 2.4>' == line:
        old_version = True
        continue

    elif '</IfVersion>' == line:
        if old_version:
            old_version = False

        continue

    elif old_version:
        continue

    elif '</IfDefine>' == line:

       skip = skip_flags.pop()

       if skip:

           skip_level -= 1

       continue

    elif defined_no == line[0:len(defined_no)]:

       key = line[len(defined_no) : -1]

       skip = True if key in env_vars else False

    elif defined_yes == line[0:len(defined_yes)]:

       key = line[len(defined_yes) : -1]

       skip = True if key not in env_vars else False

    if skip is not None:

       skip_flags.append(skip)

       if skip:
           skip_level += 1

       continue

    if skip_level == 0:

        # Compress multiple output blank lines to single blank line
        #
        if re.match(r'^\s*$', line):
            n_blank_line += 1
        else:
            n_blank_line = 0

        if n_blank_line > 1:
            continue

        print(line)

funit.close()

[Graham Dumpleton]

On 10 May 2020, at 11:39 pm, 'OwlHoot' via modwsgi <mod...@googlegroups.com> wrote:

I have used mod_wsgi-express to whump up an Apache config. But I found a problem whereby when run by user "apache" it can't find a python module which I know is present when running under "root" :

# whoami
root

# python -c "import django;"
#

# su apache

$ python -c "import django;"

  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'django'

Most likely because you installed packages as root with some strange restrictive umask or something which resulted in files installed not being accessible to non root user. That or you installed stuff under /root home directory as per user site-packages, which since /root is not accessible to other users will not be able to be read by Apache user.

I recall reading that mod_wsgi and mod_wsgi-express is installed with python running in some virtual environment. But as I am running this in a Docker container, I am really not interested in and know little about outdated old stuff like venv, virtualenv, and suchlike (and I wish there was a mod_wsgi-express option to use the system-wide python, and not bother with spinning a virtual cocoon round the python it uses, as doing this simply complicates things when using Docker! )

Using the system Python installation area rather than a Python virtual environment is still best practice in a container image, because Python packages installed by and required by the operating system can interfere with your ability to install packages you need of specific versions. For a big explanation of the problem see the post I made about this at:

* http://blog.dscpl.com.au/2016/01/python-virtual-environments-and-docker.html

So my question is how can I "devirtualise" this python that runs under user "apache", so it runs the system-wide python3 and its libraries, and thus pick up the django module among others, the same as "root"?

I really recommend against that. I also recommend against running your container as root to begin with as that is a security issue in itself. You also should drop capabilities when running the container so that su/sudo don't work. There is no need for you to start Apache as root and then switch to Apache user.

A series of best practices is:

* Add a new user account (use pid 1001).

* Create a home directory for that user and ensure that the user owns it so it can write to it.

* Switch user in Dockerfile to 1001 (don't use the username, use the uid so platforms can verify the container doesn't run as root). The container will later be run as 1001 and not as root.

* Create a virtual environment in the home directory.

* With virtual environment activated pip install mod_wsgi and all your application packages.

* Set up command to run mod_wsgi-express, use port 8080. Do not use port 80 as there is no need to. Map port 80 to port 8080 outside using docker runtime if that is what you really need exposed.

There are a bunch of other things you can do to improve this but too much to detail here.

A lot of it is perhaps out of date and doesn't reflect what I would regard as best practice now, but have a bunch of other blog posts pin related topic of running Python with Docker at:

* http://blog.dscpl.com.au/p/using-python-with-docker.html

I have other workshops as well which cover most up to date stuff.

Graham

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/92ed75db-0c73-462f-8eee-f1e563b42d3f%40googlegroups.com.

[Sujai Kumar]

In addition to what Graham sir has mentioned. You can easily see it for yourself on what is the  module search path for django when it is found as root user and as apache user using the snipped below.

Run the below as 'root' user

$ python -c "import sys;print(sys.ptah);import django;print(django.__file__)" # This will give the path from where django is retrieved.

Run the below as 'apache' user

$ python -c "import sys;print(sys.path);import django;" # This will give all the paths where python would look into to retrieve django module.

If you feel that you have read access to the apache user for the module path (of django), you could make python in apache user to include the search path by appending to sys.path

$ python -c "import sys;print(sys.path);sys.path.append('/path/where/django/is/found/');import django;" # This would help in loading the django path

Again, Please use caution from security stand point as you are allowing things that are installed for root to be used by other user.

Thanks & regards,

Sujaikumar

[OwlHoot]

On Monday, 11 May 2020 05:57:14 UTC+1, Sujai Kumar wrote:

> In addition to what Graham sir has mentioned.

..

Many thanks for your prompt replies guys. I fixed the python issue, but now Apache is returning "500 server error" for both "http" and "httpd" requests (see follow up post I just added)

Regards

John R

[OwlHoot]

Many thanks for your prompt replies guys. I fixed the python issue, but now my Apache config is rejecting with a "500 server error" both "http" and "httpd" requests (see next question I posted here just now)

Regards

John R