Multiple Django instances served using mod_wsgi with same global python interpreter

[RajKumar Ambadipelli]

Hello Graham,

I have python django microservice running on apache web server using mod_wsgi in daemon mode, It takes the request and forwards to database and takes response from database and forwards it to front-end service.

But like the above I have around 275 services and there is no cpu-bound tasks on them, they simple take request and forwards to database and gets response from database and forwards to front-end service.

So, All of these wsgi daemons virtualhost configurations have same line i.e.,
WSGIApplicationGroup %{GLOBAL}
all of them are using same python interpreter and there are no sub interpreters if I am correct.

The below is example for my one of my virtualhost config

#MyApp1 Webservice Config
Listen 9013

<VirtualHost *:9013>
        ErrorLog /var/log/webservice_error.log

        WSGIPassAuthorization On
        WSGIDaemonProcess 9013 python-path=/home/admin/myapp1:/home/admin/shared display-name=%{GROUP}
        WSGIProcessGroup 9013

        WSGIApplicationGroup %{GLOBAL}
        WSGIScriptAlias / /home/admin/myapp1/conf/wsgi.py

        <Directory /home/admin/myapp1/conf>
            <Files wsgi.py>
             Require all granted
            </Files>
        </Directory>
</VirtualHost>

and below is config of my wsgi.py

import os

from django.core.wsgi import get_wsgi_application

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'conf.settings')

application = get_wsgi_application()

Does this create any problem, Or there is any other way I can do it?
I have tried using dockers and kubernetes but for each container it is consuming more memory and cpu compare to deploying all of them in a single apache web server.
I am able to run all my services on apache web server but not on docker containers in kubernetes.

Thanking You,
RajKumar

[Graham Dumpleton]

When you say you have around 275 services, do you mean you 275 VirtualHost's and thus also 275 mod_wsgi daemon processes?

How do the services differ? Are they completely different code bases, or are they the same service for but for different customers/users distinguished by host name?

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/e508f452-1f63-46a1-ba7a-5f2b96a6739cn%40googlegroups.com.

[RajKumar Ambadipelli]

Yes all those (275) are virtualhosts like below

#MyApp1 Webservice Config
Listen 9013

<VirtualHost *:9013>
        ErrorLog /var/log/webservice_error.log

        WSGIPassAuthorization On
        WSGIDaemonProcess 9013 python-path=/home/admin/myapp1:/home/admin/shared display-name=%{GROUP}
        WSGIProcessGroup 9013

        WSGIApplicationGroup %{GLOBAL}
        WSGIScriptAlias / /home/admin/myapp1/conf/wsgi.py

        <Directory /home/admin/myapp1/conf>
            <Files wsgi.py>
             Require all granted
            </Files>
        </Directory>
</VirtualHost>

and below is config of my wsgi.py

import os

from django.core.wsgi import get_wsgi_application

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'conf.settings')

application = get_wsgi_application()

I have for 275 let's say myapp1... ,to myapp275 and each is running on a seperate wsgi daemon with same python interpreter.

And all of them are different services.
And all of them are having different code base.

[Graham Dumpleton]

So your main concern if this setup is working is how to reduce memory usage if possible? Yes/No?

How many actual URL handlers would each service have, and how many requests/sec would each be handling? Are they all services used quite frequently, or are some/many use quite infrequently?

To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/e4362b25-6078-480a-9abd-d5111aa4fd8cn%40googlegroups.com.

[RajKumar Ambadipelli]

Yes,  My main concern is 
The above what I have mentioned environment is working fine in my development environment but is this method is feasible in production server or not.

And coming to my URL handlers my each application served using different wsgi daemon is having and average of 70 URL handlers.

And services are not used that frequently most of them are used infrequently.

And I have not tested the traffic load till now i.e., requests/sec and my application is not yet moved to production.

[Graham Dumpleton]

Sorry, one more question for now while think of what to suggest.

Are these REST API type services, HTML page based URL handlers, or mixed of both?

To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/e4362b25-6078-480a-9abd-d5111aa4fd8cn%40googlegroups.com.

[RajKumar Ambadipelli]

Only REST API type services.

[Graham Dumpleton]

And more questions.

What Apache MPM are you using? Prefork, Worker or Event.

How much memory are the Django web application processes using? IOW, the mod_wsgi daemon processes.

To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/4bb11732-f75c-4e0a-bfe2-73c485fc09edn%40googlegroups.com.

[RajKumar Ambadipelli]

Each mod_wsgi is using an average of 7 to 10mb of ram. And coming to cpu usage i got to see spike in cpu only while forwarding request or while sending response to client.

And apache mpm: mpm_event_module

[RajKumar Ambadipelli]

And spike in cpu while forwarding request to datababse or sending response to client is in between 30 to 35% of a single core.

[Graham Dumpleton]

And it is a single database shared by all front ends right?

To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/3c0865ba-c36f-43f2-bef9-5562964918ban%40googlegroups.com.

[RajKumar Ambadipelli]

No databases were different

[Graham Dumpleton]

Let me rephrase that then, one database server, but with separate database instances hosted by that server?

Or do you really have 275 completely separate database servers?

[RajKumar Ambadipelli]

I have 5 databases running on different ports i.e., seperate instances and all 275 services running on single apache web server will connect to these databases