Apache 2.4 running as a Windows Service

[Bruce Gibbins]

Hi,

I realise this will probably cause me some backlash from the community but I have an issue which I am sure is environmental but have been struggling for a few days to resolve. 

The back story is that I have successfully had Apache 2.4.37 installed on our server running mod_wsgi supporting an internal Django solution we are working on for quite some time.

Apache is running as a service on this server. Python was/is deployed via Anaconda virtual environment.The site has been configured as a Virtual Host

Everything works well. During the project life we cloned the setup into a DEV set of directories and created a second Windows Service to support that second environment.  Nothing was shared in the sense that everything was setup and isolated in their own setr of directories. 

The only thing that was shared was Python via the Virtual Environment. mod_wsgi was PIP installed into here after ensuring all of the appropriate VC libraries had been installed. 

Apache 2.4.37 VC15 x64 was installed from an ApacheLounge binary distro.

Everything as far as i can tell is x64

Earlier this month I decided it was time to upgrade Django from 2.1.7 to 3.x. Our Apps are fairly basic and the whole environment pretty much out-of-the-box.

I created a new virtual environment and brought it up to Python 3.9 and upgraded Django and all of its dependencies at the same time. There were some challenges with some dependencies but I eventually got them all sorted.

I did as we did before and cloned the whole Apache folder, adjusted httpd.conf and httpd_hosts.conf which was typically just changing the various path names. 

After stopping the existing DEV service I fired up httpd.exe from the command line after changing my working directory to the virtual environment hosting the updated python and Django site-packages. After some initial config issues - have it all running. The PROD service continues to run as well, pointing to the production version of the site using the older version of Django and site-packages.

As the existing Apache DEV service I had previously created affectively points to all of these directories and nothing has changed in how it starts. I simply left it as is.

The service will not start. When reviewing one of the logs I discovered what I believe to be the issue but I don't understand how to correct it as I believe I have done everything required for setting up mod_wsgi in embedded mode on Windows.

The error text indicates to me that the Python standard library .DLLs and .Lib directories are not correctly discovered as they indicate ".\\DDLs" and ".\\Lib". The sys.base_prefix is correct. Because of this I then get the import error. I don't believe wsgi.py even gets a chance to fire up properly.

I know there is a lot to consume here. But I have also included the relevant section from the httpd.conf and httpd_vhosts.conf files

    WSGIApplicationGroup %{GLOBAL}

    WSGIScriptAlias / "E:/tgm/sites/dev/django-agora/agora/wsgi.py"

    <Directory E:/tgm/sites/dev/django-agora/agora>

        <Files wsgi.py>

            Require all granted

        </Files>

    </Directory>

ERROR LOG

Fri Jun 11 07:33:42.249667 2021] [wsgi:info] [pid 15408:tid 592] mod_wsgi (pid=15408): Initializing Python.

Python path configuration:

  PYTHONHOME = (not set)

  PYTHONPATH = (not set)

  program name = 'python'

  isolated = 0

  environment = 1

  user site = 1

  import site = 1

  sys._base_executable = 'E:\\tgm\\bin\\Apache24-DEV\\bin\\httpd.exe'

  sys.base_prefix = 'E:\\tgm\\bin\\miniconda3\\envs\\agora-v3'

  sys.base_exec_prefix = 'E:\\tgm\\bin\\miniconda3\\envs\\agora-v3'

  sys.platlibdir = 'lib'

  sys.executable = 'E:\\tgm\\bin\\Apache24-DEV\\bin\\httpd.exe'

  sys.prefix = 'E:\\tgm\\bin\\miniconda3\\envs\\agora-v3'

  sys.exec_prefix = 'E:\\tgm\\bin\\miniconda3\\envs\\agora-v3'

  sys.path = [

    'E:\\tgm\\bin\\miniconda3\\envs\\agora-v3\\python39.zip',

    '.\\DLLs',

    '.\\lib',

    'E:\\tgm\\bin\\Apache24-DEV\\bin',

  ]

Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding

Python runtime state: core initialized

ModuleNotFoundError: No module named 'encodings'

HTTPD.CONF

LoadFile "e:/tgm/bin/miniconda3/envs/agora-v3/python39.dll"

LoadModule wsgi_module "e:/tgm/bin/miniconda3/envs/agora-v3/lib/site-packages/mod_wsgi/server/mod_wsgi.cp39-win_amd64.pyd"

WSGIPythonHome "e:/tgm/bin/miniconda3/envs/agora-v3"

HTTPD-VHOSTS.CONF

    WSGIApplicationGroup %{GLOBAL}

    WSGIScriptAlias / "E:/tgm/sites/dev/django-agora/agora/wsgi.py"

    <Directory E:/tgm/sites/dev/django-agora/agora>

        <Files wsgi.py>

            Require all granted

        </Files>

    </Directory>

Thank-you in advance and again my apologies if I have over detailed things and not followed some specific requirements for the Windows environment. I am just after some hints as to what may lead to the incomplete paths being pushed into sys.path 

Regards

Bruce

[Graham Dumpleton]

Your description is a little bit confusing. You said DEV was shutdown and PROD started, which was working fine. You then said DEV was left as it is, which I presume you mean shutdown. Then you said the service doesn't start.

The service for what instance doesn't start?

A few warnings.

Anaconda Python is a really bad choice for using in embedded systems. It keeps breaking guarantees around Python embedding and so often versions of it will not work in embedded systems. I always recommend people avoid it when using mod_wsgi.

Next, a system service in Windows still needs to have the required permissions to access any Python virtual environment, plus application code. If it can't access the Python virtual environment, you can get errors like you see.

Graham

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/43d2c4f0-ef9b-443b-a28f-4ac52fc213c7n%40googlegroups.com.

[Bruce Gibbins]

Thanks Graham. yes I agree it was probably a bit confusing and difficult to explain without some screen shots. I will see if I can add those.

But basically, I initially had (all working) Two Services one being the default Apache HTTP Server running out of e:\tgm\bin\apache24. This was meant to be our PRODuction instance. I then cloned this set of directories into another set called e:\tgm\bin\apache24-dev. i then created a second service from here and called it Apache HTTP DEV Server. Nothing in PROD has been touched. It is still running and using Python 3.6 and Django 2.1.7 with a matching mod_wsgi pip installed at the time.

All of these were configured with their own set of .conf and vhosts.conf files and I used  mod_wsgi-express to generate the module configuration lines. All of the paths were pointing to the Virtual Environment homes. These all work and fire up without issue.

Affectively, all I have done now is stopped the DEV service instance. Created a new virtual env and brought all of the packages up to near current versions to suit Django 3.x. used pip install mod_wsgi in that new environment hopefully match up the python version with mod_wsgi. I have then adjusted the DEV HTTPD .conf files to use the new paths.

All of this works when I run httpd.exe from the command line starting from the new virtual environment root (where in this case python 3.9 lives) - Again all functional, new Django is loaded, website and web app all functional. We even have ADFS in play and that all hangs together. 

However, as soon as I restart the DEV HTTP Server service I get the errors and what appears to be a missing path the Lib. If I change the .CONF files to point back to the previous python 3.6 virtual environment, the service starts and we have our DEV platform back - just at the older versions.

All permissions are ok and the service account can reach the virtual environment.

I apologise that my technical skills don't go into the depths of WSGI and its ecosystem and I am probably confusing things. I was expecting some challenges - but I am almost there but just can't seem to pick why the Lib and DLLs paths are incomplete.

I have not had any experience with the other venv solutions and was just hoping to basically upgrade what we had already working.

Regards

bruce

[Bruce Gibbins]

Hello Graham

I realise you have probably established that there is not much more you can offer as advice. But can you shed any light on HOW the elements of sys.path are built based on the screen shot I have attached. I am going to try using VENV instead of CONDA but in a last ditch effort to understand why it is using the current directory for DLLS and Lib when all indications are that it should find them based on sys.base_prefix which is pointing to the correct directory. Knowing this may provide me with some insight as to why my first Virtual Environment using Python 3.6.5 is working as expected when I start HTTPD as a service.

Based on the log in the image it appears to me that mod_wsgi is the last element called, it finds an issue (probably evoking wsgi.py) and then shiows this detail.

[Graham Dumpleton]

I did warn you that Anaconda Python keeps breaking the ability to be used in embedded applications, especially when working with virtual environments. If you are 100% sure that all permissions are correct such that the service can access it, which I am still not convinced they are if it works from command line, all I can suggest is you create a configuration which doesn't use a virtual environment and points at a WSGI script file which has the most basic WSGI hello world, with no external packages used. Eg.,

def application(environ, start_response):
    status = '200 OK'
    output = b'Hello World!'

    response_headers = [('Content-type', 'text/plain'),
                        ('Content-Length', str(len(output)))]
    start_response(status, response_headers)

    return [output]

See if even that works.

Graham

On 13 Jun 2021, at 7:53 am, Bruce Gibbins <bruce....@gmail.com> wrote:

Hello Graham

I realise you have probably established that there is not much more you can offer as advice. But can you shed any light on HOW the elements of sys.path are built based on the screen shot I have attached. I am going to try using VENV instead of CONDA but in a last ditch effort to understand why it is using the current directory for DLLS and Lib when all indications are that it should find them based on sys.base_prefix which is pointing to the correct directory. Knowing this may provide me with some insight as to why my first Virtual Environment using Python 3.6.5 is working as expected when I start HTTPD as a service.

Based on the log in the image it appears to me that mod_wsgi is the last element called, it finds an issue (probably evoking wsgi.py) and then shiows this detail.

To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/bb88f63d-13eb-4816-bdfa-13530a162ab4n%40googlegroups.com.
<2021-06-13 07_44_36-Clipboard.png>

[Bruce Gibbins]

Thanks Graham.

Won't bother you any further unless I find a solution and I have taken on board your advice and will move to that alternative environment. I am only pursuing this as I would like to understand more about why as it may help in the future.

I also totally understand about the permissions and they may in fact be at play and I just can't see it. The account I am using to run it as a service and to run it from the command line are one in the same - a specifically created domain account . It is also the account used to run the production version. The command line is being run as Administrator and that may be affecting things. 

But in any case, I will take your advice and look at a non-conda based venv.

Thanks again for your help and advice.

Kind Regards

Bruce

To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/4EC15964-8F33-4B64-A804-C27779F4258E%40gmail.com.

[Bruce Gibbins]

Hi.

For anyone who stumbles across this post I thought I would mention what has worked for me - with thanks to Graham for his time and advice.

Anaconda or in my case Miniconda did work and was generally a good fit for us and our internal deployment strategies for other Python based solutions including Django. However, with recent updates to many of these libraries we were no longer able to get Apache to run as a service when hosting these internal Django sites and webapps. Graham was 100% correct in that Conda seems to 'pollute' the environment in its favour and as such does not always play well with the neighbours.

In summary and missing out server and directory specifics the general solution that worked for me was to :

- Install latest python into its own base directory with (eg. C:\bin\python\v3.9.5).

- Already had the Microsoft Visual C++ redistribution libraries installed.

- Created a separate area to host the virtual environment projects (something like eg c:\projects\env) .

- I then used Python venv to create a new site virtual environment within this parent directory.

- Activated this venv.

- PIP Installed all of the Django and site dependencies bringing them all up to new/current versions.

- PIP installed mod-wsgi into this venv.

- From the Django WebSite project directory I ran "python migrate.py check" to make sure it was generally healthy. In my case I needed to fiddle a few Django settings but otherwise it was all good.

- ran "mod_wsgi-express module-config" to get the config elements for the Apache httpd.conf file.

- tested all of this by running httpd.exe from within the active venv.

- stopped the command instance of httpd.

- As I was reusing an existing Apache install as per the previous threads in the post I started the windows service version which I had already previous created prior to all of this.

- The service ran without too many issues,. There were a couple of local environmental things I needed to do but nothing that could not be figured out from the various log files and typically Django related.

As we run a DEVelopment/TESTing environment as well as a PRODUCTION version on the same server due to the lightweight nature of our project I have 2 Apache instances one for DEV and one for PROD. After testing in DEV I then simply updated the PROD httpd.conf file using the same mod_wsgi settings used for DEV as they are going to share the same Python venv. Down the track I will create a separate DEV and PROD venv so that we can keep the underlying packages etc isolated and then update the Python and Django libraries in isolation. At the moment my focus was to just get both environments running Django v3+ under Apache as a service.

I know that there are many elements of detail I have left out but they are generally specific to us and the summary coupled with Graham/'s documentation and the fundamental move away from Anaconda enabled us to get the upgrade back up and running.

regards

Bruce