Host multiple Django websites, one Apache process, on Windows

[SC]

Firstly, I'd like to thank you for maintaining this excellent utility.

I should note that I am not a technically-versed person in these matters, so apologies if I am slow to pick up on things.

As for my problem, I want to host multiple Django-based websites from the same httpd.exe process on Windows. I do not know how to go about this.

Details on my setup:

- mod_wsgi (v5.0.0)

- Python (v3.9.2)

- Apache (v2.4.58), with the binaries being distributed by ApacheLounge (2024-01-31 release)

- Windows Server 2012 R1 (and not R2)

- Django v4.2.10

- I am hosting two distinct domain names (e.g. "someexample.com" and "anotherexample.com")

- Everything is set up from scratch in the sense that I have physical access to the machine running the web server, access to the domain names, access to the router, etc.

- Because I am running Windows, I am running mod_wsgi in embedded mode. My installation of Apache is mostly standard, so something like mod_python is not installed.

Here is the relevant snippet of Apache configuration. I am aware that it does not run (because WSGIPythonPath cannot be in a VirtualHost directive), but it encapsulates the logic of what I am trying to go for:

LoadFile "C:/Users/Administrator/AppData/Local/Programs/Python/Python39/python39.dll"
LoadModule wsgi_module "C:/Users/Administrator/Desktop/SERVERS/web_server_production/python_virtual_environment/lib/site-packages/mod_wsgi/server/mod_wsgi.cp39-win_amd64.pyd"

WSGIPythonHome "C:/Users/Administrator/Desktop/SERVERS/web_server_production/python_virtual_environment"

# VIRTUAL HOST: "someexample.com"

<VirtualHost *:80>
ServerName someexample.com
ServerAlias www.someexample.com
Redirect permanent "/" "https://someexample.com/"
</VirtualHost>

<VirtualHost *:443>
ServerName someexample.com
ServerAlias someexample.com
<If "%{HTTP_HOST} == 'www.someexample.com'">
Redirect "/" "https://someexample.com/"
</If>

SSLEngine On
SSLCertificateFile "..."
SSLCertificateKeyFile "..."

WSGIScriptAlias / "C:/Users/Administrator/Desktop/SERVERS/web_server_production/website_projects/someexample.com/main/wsgi.py"
WSGIPythonPath "C:/Users/Administrator/Desktop/SERVERS/web_server_production/website_projects/someexample.com"

<Directory "C:/Users/Administrator/Desktop/SERVERS/web_server_production/website_projects/someexample.com/main">
<Files "wsgi.py">
Require all granted
</Files>
</Directory>

Alias "/assets/" "C:/Users/Administrator/Desktop/SERVERS/web_server_production/website_projects/someexample.com/@assets/"

<Directory "C:/Users/Administrator/Desktop/SERVERS/web_server_production/website_projects/someexample.com/@assets/">
Require all granted
</Directory>

</VirtualHost>


# VIRTUAL HOST: "anotherexample.com"

<VirtualHost *:80>
ServerName anotherexample.com
ServerAlias www.anotherexample.com
Redirect permanent "/" "https://anotherexample.com/"
</VirtualHost>

<VirtualHost *:443>
ServerName anotherexample.com
ServerAlias www.anotherexample.com
<If "%{HTTP_HOST} == 'www.anotherexample.com'">
Redirect "/" "https://anotherexample.com/"
</If>

SSLEngine On
SSLCertificateFile "..."
SSLCertificateKeyFile "..."

WSGIScriptAlias / "C:/Users/Administrator/Desktop/SERVERS/web_server_production/website_projects/anotherexample.com/main/wsgi.py"
WSGIPythonPath "C:/Users/Administrator/Desktop/SERVERS/web_server_production/website_projects/anotherexample.com"

<Directory "C:/Users/Administrator/Desktop/SERVERS/web_server_production/website_projects/anotherexample.com/main">
<Files "wsgi.py">
Require all granted
</Files>
</Directory>

Alias "/assets/" "C:/Users/Administrator/Desktop/SERVERS/web_server_production/website_projects/anotherexample.com/@assets/"

<Directory "C:/Users/Administrator/Desktop/SERVERS/web_server_production/website_projects/anotherexample.com/@assets/">
Require all granted
</Directory>

</VirtualHost>

If further information is needed, such as what my file structure for these website (Django) projects look like, or how wsgi.py is configured (I mostly used the default generation given by Django), I can provide those too.

Probably worth noting, but when starting a new project with Django, it usually names the "main package" as whatever the project was named (usually the name is just the website's name). Instead, I renamed that package to "main" (and it has the same name across all websites) and updated the appropriate references (e.g. "main.settings" inside of "wsgi.py" instead of "someexamplecom.settings"). This is a convention I would prefer to keep, if possible, though I am willing to revert to Django's default convention if it is truly necessary.

[Graham Dumpleton]

Have a read of:

* http://blog.dscpl.com.au/2012/10/requests-running-in-wrong-django.html

and see if it answers your questions.

Most likely since using Django any problems will be due to how DJANGO_SETTINGS_MODULE is set. The default that project template generates causes problems and you need to change it.

Graham

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/modwsgi/d2b81c58-9d22-4720-a6d3-38eec28ca9a8n%40googlegroups.com.

[SC]

Thanks for the article. I followed through on making that one change to my wsgi.py configuration, and (after making two other changes) things are working on my end.

One of the other changes was to move WSGIPythonPath outside of the virtual hosts, then use both paths within the same directive (separated by a semicolon).

However, there is one point of a lack of understanding on my part that's bothering me. What exactly is an environment variable, and why is it even needed? (Why not just read a value from a text file? - I understand these questions are on the topic of Django moreso than WSGI, so if it's too out of scope, then I'll bring it up over there rather than here.) I'm asking because each one of my website projects has the following file structure:

website-name-here1.com

|

---- manage.py

|

---- main

|

---- app1

|

---- app2

|

...

Normally, when running Django's "startproject" command as per their tutorial, you get:

websitenamehere1com

|

---- manage.py

|

---- websitenamehere1com

|

---- app1

|

---- app2

|

...

Personally, I found the semantics of this default way of naming things unacceptable (to my mind), so I changed it to the above. This is a convention I would strongly prefer to keep. However, this ended up with the very first Django instance running fine, and the second Django instance throwing 400 "Bad Request" errors to the client.

More specifically, if I configured WSGIPythonPath to "C:/.../website_projects/example1.com;C:/.../website_projects/example2.com", it will be example2.com that throws 400 errors. If I reverse the order of that configuration, it will be example1.com that throws 400 errors. Both website-projects had their folder named "main" (i.e. the same name), so naturally, both of their wsgi.py files had:

os.environ['DJANGO_SETTINGS_MODULE'] = 'main.settings'

While I had already suspected that this could cause problems (reusing the same name, "main") when running multiple instances, I do not have a detailed understanding of why. I thought that we would have one Apache process spawning two Python processes (I should note that I have next to no understanding of systems programming, Apache, WSGI, etc.), and each Python process would have its own local set of environment variables and look only within its own directory (as specified in WSGIPythonPath). It could inexplicably be the case that setting the environment variable, "DJANGO_SETTINGS_MODULE", is global, and therefore sets the value "main.settings" for both Django instances, but even if this were the case, it shouldn't matter. (I.e. they both use the same value for that variable - unless it somehow gets resolved behind the scenes to an absolute filepath.) So it might instead have to do with WSGIPythonPath being confused as to which "main.settings" it should be picking from the two paths that I configured for it.

Either way, while I have a "fix", I would like to understand what is going on here and whether the above convention can be kept with some workaround.

Also, off-topic; the current docs state that the WSGIPythonHome directive is not available on Windows, but it appears to be required to have around, or else Django will throw an error.

[Graham Dumpleton]

WSGIPythonHome may indeed work on Windows now. It wasn't originally from memory as the Python C APIs to support it on Windows were broken. I use a hack to make it work now I think.

Environment variables are a thing mainly used on UNIX systems rather than on Windows. You don't have to use them to inject config into applications, but I think that is the only way Django supports DJANGO_SETTINGS_MODULE.

Anyway, try not using WSGIPythonPath. Instead, at the start of the WSGI script file, what you reference using WSGIScriptAlias, set the Python module path explicitly in Python code before anything else.

Thus one script file would use:

import sys, os

sys.path.insert(0,  "C:/.../website_projects/example1.com")

os.environ["DJANGO_SETTINGS_MODULE"] = "main.settings"

and the other:

import sys, os

sys.path.insert(0,  "C:/.../website_projects/example2.com")

os.environ["DJANGO_SETTINGS_MODULE"] = "main.settings"

Follow this will original contents of the file.

This way the Python module path changes are specific to each Python sub interpreter context and not setting it globally such that all inherit the same.

Graham

To view this discussion visit https://groups.google.com/d/msgid/modwsgi/39ecdbd5-5e04-4269-96df-86a998936e11n%40googlegroups.com.

[SC]

Hi there - sorry for the very long delay in between posts.

Firstly: I had followed your instructions and got my multi-website setup to work in my testing environment. I have yet to try it in a production environment, but I am sure it should work there too. Thanks for the speedy and concise help you have given thus far.

I did, however, have a few additional questions surrounding the workings of Apache and mod_wsgi, at the time that I read and implemented your suggestion (i.e. in early February). This is largely because while getting results from following advice is fine, I wanted to be sure I really understood what I was working with, and why things (such as particular configuration snippets) work the way they do. I could have asked vague and probably malformed questions at that time, but decided instead to spend some time studying systems programming and looking over Apache/mod_wsgi documentation in order to ask better questions, hence the long time delay between now and my previous post.

Now, regarding the distinction between "embedded mode" and "daemon mode", I take the main difference to consist in how the Python interpreter is executed from the perspective of the operating system. (I am aware that my setup is Windows and therefore supports only embedded mode - I am asking this more for understanding what I am working with rather than getting some specific thing to work.)

In "embedded mode", the Apache worker process has the Python interpreter running "inside of it" - perhaps as its own thread. (In other words, whatever the Apache worker natively does is one thread, and the Python interpreter is another.) Whereas in "daemon mode", the worker process requests the OS to spin up a new process that is reserved for the Python interpreter to run in (which will then presumably execute WSGI applications). In the case of "embedded mode", I am having some trouble understanding what it means for a Python interpreter to run inside of a (pre-existing) process, such as an Apache worker. It may also be worth mentioning that the details of what exactly is involved with an Apache worker process and what it does is still opaque to me at the moment.

The above confusion is related to my next question. In the mod_wsgi documentation (and in other places such as PEP 684), there are mentions of "sub-interpreters", which I am also having trouble conceptualising, even after reading resources about it online. What does it mean for a single process to host multiple sub-interpreters? (This is really what I want to understand for my own setup.) Are these sub-interpreters instantiated as threads within the process - i.e. each sub-interpreter is its own thread? If so, what within the process is coordinating these various threads (i.e. sub-interpreters), and what exactly is shared between these sub-interpreters?

I am making the assumption of viewing this purely from the perspective of the operating system (and not from, say, the programmer working on a WSGI application). Hence I assume that the only thing that really exists are the processes that are defined by the OS. Consequently, how programs are threaded, and the logic of how threads are coordinated, is left to the logic of the various programs running inside of the process.

To relate an infographic on sub-interpreters that added to my confusion, here it shows what is presumably a single process used to run multiple Python sub-interpreters. The language suggests that each sub-interpreter can host multiple threads, but that each sub-interpreter is itself somehow not a thread (despite presumably belonging to the same process, which hosts the global state storage that all sub-interpreters share). Additionally, I am unclear on what the relation is between the main interpreter and subsequently-spawned sub-interpreters, along with how the process coordinates between these interpreters; I am assuming that, possibly, the main interpreter does the work of coordinating between which sub-interpreters run at which time (e.g. after exhausting their allotted quanta).

The main point is that I am used to associating a Python interpreter with a process (e.g. seeing "python.exe" show up in the list of processes). I am neither familiar with the notion of what it means to run the interpreter inside of another pre-existing process (that's already running something else, like an Apache worker), nor what it means for a process to contain sub-interpreters. If any of my questions/concerns are unclear, I can try to rephrase them better.

[Graham Dumpleton]

Quick reply on just the main point of confusion.

The Python interpreter does not run in its own thread.

The Apache code and Python code are running within the same process. The Apache code is the main entry point and creates a pool of threads which are handling incoming web requests. When one of these threads accepts a web request and it is determined that it is for the Python WSGI application, the thread calls into the mod_wsgi code, which in turn calls into the Python interpreter and executes the Python code.

Being a thread pool, multiple web requests could be getting handled at the same time and running concurrently, thus resulting in multiple calls into your Python code from the different threads of the thread pool.

I can't remember exactly what is covered in them, but some of these talks may help a little bit in understanding things, but also may confuse things for you.

To view this discussion visit https://groups.google.com/d/msgid/modwsgi/3ac445cc-8fe8-4dd6-8321-5ed2fc0e1266n%40googlegroups.com.

[SC]

Thanks for posting the videos. While they don't directly address my questions, they did contain a lot of useful information. I've also done quite a bit of digging in other resources and I think I should be able to frame my thoughts more clearly now.

For context, I am not really a developer nor an admin and have never compiled a thing in my life - my motivation for going through all this is to thoroughly document and explain an entire system setup and its data flow.

I'll lay out what I think is going on to the best of my ability, and then maybe we can see where the gaps in my understanding are:

• Apache for Windows is a binary built by compiling a bunch of C source code files and statically linking them together (i.e. lumping them into the same binary). One of the statically-linked modules of importance is mpm_winnt.
• Other modules, built from C source code, can instead be dynamically linked. They are not part of the Apache binary, but the LoadModule directive can load them into memory and dynamically link shared object files (e.g. .so, .dll, .pyd) to the Apache control (i.e. parent) process that is in memory. One such relevant module is mod_wsgi (for my setup, built against CPython3.9 for Windows using an AMD64 processor).
  
• When executing httpd.exe, the Apache control process is created. This process has the general capabilities of both statically-linked and dynamically-linked modules (e.g. mpm_winnt and mod_wsgi). This control process has two general responsibilities: listening to an IP address range and a port number (i.e. a network interface), and spawning, deleting, managing, and forwarding HTTP requests to worker processes for them to handle.
  • Because I am using mpm_winnt, it will only create 1 worker process with 150 threads as per default configuration. This worker process also contain all the modules that were contained in the control process, including mod_wsgi.
    

• With HTTP requests handled the "old-fashioned" way, the worker process spends most of its (CPU) time looking for a file on the system, bundling it into the body of an HTTP response, and building the rest of the HTTP response. The 150 threads of this lone worker dictate how many requests it can start work on (and of course, there is the practical problem of backlog and such, but that's out of the current scope).
  • These threads are based on a C-based threading library, with quite different behaviours from Python's GIL-bound threading library. One difference is that these 150 threads can potentially make use of multiple processor cores*.
  • *Based on watching your videos, this doesn't seem to be the case. I can't say I quite understand threading in practise - I'm too used to Python's version of threading, and I've also heard that threading libraries in C expose the threads to the operating system process manager. This should be left as a separate topic, however.
    
• The worker process has mod_wsgi in its memory. The worker process instantiates the Python main interpreter using the CPython API, via Py_Initialize(). This main interpreter is now in its memory, ready to be called any time from any context (i.e. any thread). The main interpreter is not bound to memory that is specific to any of the 150 threads: therefore, it can be invoked from any individual thread - this memory space is common to all threads.
  

• I have two website projects (both Django), two virtual hosts, and two WSGIScriptAliases (one for each virtual host). Some requests will be destined for one website, and other requests to the other website, based on their HOST header.

• A request is received by the Apache control process with a particular HOST header. It forwards it to the worker process, and selects one of its unoccupied threads at random/pseudorandom. The worker process (whenever we are within this thread's quantum) decides to invoke the Python main interpreter (sitting in thread-neutral memory) and tells it to create a Python sub-interpreter (owned and managed by the main interpreter, but also sitting in thread-neutral memory). Then, it tells this (named) Python sub-interpreter to invoke the appropriate wsgi.py script.
• The entirety of this wsgi.py script is handled by this Python sub-interpreter, which is now blocking this thread. This thread will wait for this Python sub-interpreter to return the data needed to create an HTTP response object. Once the sub-interpreter returns data, the C code in this thread can continue on, e.g. make an HTTP response and pass it back to the requesting user agent. The named sub-interpreter is now associated with this website, and will persist in memory.
  
  • This wsgi.py script (running in a Python venv) just contains "pointers" to Django and related Django modules, like settings.py, in effect having Django process the HTTP request data to generate and return the appropriate HTTP response data.
  • Conveniently, we have a sub-interpreter associated with this website (i.e. WSGI script): various libraries and Django modules will also persist within the memory of this sub-interpreter.
    

• Another HTTP request is passed to the same virtual host. A random unoccupied thread takes up this request. It invokes the main interpreter, but sees that the named sub-interpreter it was looking for already exists. Instead of creating a new one, the thread tells the interpreter to tell the sub-interpreter to run the correct wsgi.py. Django is already loaded in the memory belonging to this sub-interpreter, so we do not have to load these Python libraries again.
• An HTTP request is sent to the other virtual host. No (named) sub-interpreters for this exists, so we have to create a new persistent sub-interpreter in memory, and a new copy of Django & Co. in this sub-interpreter's memory space.
• All threads in the worker process, from now on, will rely on this shared memory space of the main interpreter, the two sub-interpreters, and the two "Python setups".
  

Sorry if this taking a lot of your time. Finding clear information online has generally been hellish (most of it is undocumented or inadequately so), and I don't really know where else to ask (similar queries in StackOverflow usually fall flat).

[Graham Dumpleton]

On 26 Mar 2025, at 12:28 pm, SC <stvn...@gmail.com> wrote:

Thanks for posting the videos. While they don't directly address my questions, they did contain a lot of useful information. I've also done quite a bit of digging in other resources and I think I should be able to frame my thoughts more clearly now.

For context, I am not really a developer nor an admin and have never compiled a thing in my life - my motivation for going through all this is to thoroughly document and explain an entire system setup and its data flow.

I'll lay out what I think is going on to the best of my ability, and then maybe we can see where the gaps in my understanding are:

• Apache for Windows is a binary built by compiling a bunch of C source code files and statically linking them together (i.e. lumping them into the same binary). One of the statically-linked modules of importance is mpm_winnt.
• Other modules, built from C source code, can instead be dynamically linked. They are not part of the Apache binary, but the LoadModule directive can load them into memory and dynamically link shared object files (e.g. .so, .dll, .pyd) to the Apache control (i.e. parent) process that is in memory. One such relevant module is mod_wsgi (for my setup, built against CPython3.9 for Windows using an AMD64 processor).
  
• When executing httpd.exe, the Apache control process is created. This process has the general capabilities of both statically-linked and dynamically-linked modules (e.g. mpm_winnt and mod_wsgi). This control process has two general responsibilities: listening to an IP address range and a port number (i.e. a network interface), and spawning, deleting, managing, and forwarding HTTP requests to worker processes for them to handle.
  • Because I am using mpm_winnt, it will only create 1 worker process with 150 threads as per default configuration. This worker process also contain all the modules that were contained in the control process, including mod_wsgi.

I don't know if Windows is different but on non Window systems the master process only deals with configuration loading and process management. It is the worker processes which listen on the HTTP socket and accept requests, not the master.

• With HTTP requests handled the "old-fashioned" way, the worker process spends most of its (CPU) time looking for a file on the system, bundling it into the body of an HTTP response, and building the rest of the HTTP response. The 150 threads of this lone worker dictate how many requests it can start work on (and of course, there is the practical problem of backlog and such, but that's out of the current scope).
  • These threads are based on a C-based threading library, with quite different behaviours from Python's GIL-bound threading library. One difference is that these 150 threads can potentially make use of multiple processor cores*.
  • *Based on watching your videos, this doesn't seem to be the case. I can't say I quite understand threading in practise - I'm too used to Python's version of threading, and I've also heard that threading libraries in C expose the threads to the operating system process manager. This should be left as a separate topic, however.
    
• The worker process has mod_wsgi in its memory. The worker process instantiates the Python main interpreter using the CPython API, via Py_Initialize(). This main interpreter is now in its memory, ready to be called any time from any context (i.e. any thread). The main interpreter is not bound to memory that is specific to any of the 150 threads: therefore, it can be invoked from any individual thread - this memory space is common to all threads.
  

• I have two website projects (both Django), two virtual hosts, and two WSGIScriptAliases (one for each virtual host). Some requests will be destined for one website, and other requests to the other website, based on their HOST header.

• A request is received by the Apache control process with a particular HOST header. It forwards it to the worker process, and selects one of its unoccupied threads at random/pseudorandom. The worker process (whenever we are within this thread's quantum) decides to invoke the Python main interpreter (sitting in thread-neutral memory) and tells it to create a Python sub-interpreter (owned and managed by the main interpreter, but also sitting in thread-neutral memory). Then, it tells this (named) Python sub-interpreter to invoke the appropriate wsgi.py script.
• The entirety of this wsgi.py script is handled by this Python sub-interpreter, which is now blocking this thread. This thread will wait for this Python sub-interpreter to return the data needed to create an HTTP response object. Once the sub-interpreter returns data, the C code in this thread can continue on, e.g. make an HTTP response and pass it back to the requesting user agent. The named sub-interpreter is now associated with this website, and will persist in memory.
  
  • This wsgi.py script (running in a Python venv) just contains "pointers" to Django and related Django modules, like settings.py, in effect having Django process the HTTP request data to generate and return the appropriate HTTP response data.
  • Conveniently, we have a sub-interpreter associated with this website (i.e. WSGI script): various libraries and Django modules will also persist within the memory of this sub-interpreter.
    

• Another HTTP request is passed to the same virtual host. A random unoccupied thread takes up this request. It invokes the main interpreter, but

It is mod_wsgi C code which deals with the request after it is passed up from Apache C code. This C code executes independent of Python interpreter context and makes a decision to call into either main interpreter or sub interpreter context. So it doesn't pass through the main interpreter to get to a sub interpreter.

To view this discussion visit https://groups.google.com/d/msgid/modwsgi/58d4d393-9960-45ab-be5f-bafcecef0642n%40googlegroups.com.

[SC]

From what I can tell, the Windows mpm should have mostly the same logic as the worker mpm (besides being stuck at only one process).

That clears up all my questions, thanks!