rpy2 package imports are not working with "Apache Lounge(2.4) + mod_wsgi(4.6.6) + python 3.6.5 configuration" on windows machine

[Nasim Ali]

Hello All,

I've been struggling for a month now to fix this issue and needed your help.

Apache Lounge(2.4) + mod_wsgi(4.6.6) + python 3.6.5 configuration is working pretty nice on my windows machine. Thanks to 'Graham Dumpleton blogs/posts' for that.
I'm facing one strange issue which is related to rpy2 package.

Just to give you the context that till now what I have done are as follows:

1. After setting up all the required environment I ran command -> mod_wsgi-express module-config

which gives the following scripts:

LoadFile "c:/worksetup/anaconda3/python36.dll"

LoadModule wsgi_module "c:/worksetup/anaconda3/lib/site-packages/mod_wsgi/server/mod_wsgi.cp36-win_amd64.pyd"

WSGIPythonHome "c:/worksetup/anaconda3"

Added above scripts as it is in httpd.conf  file without any changes as suggested by Graham Dumpleton himself.

2. After that I've created a virtual host to run my wsgi application on 'localhost:58188/wsgi'

httpd-vhosts.conf :

 <VirtualHost *:58188>

    ServerName localhost

    LogLevel info

#WSGIProcessGroup test_optimization_group

    #WSGIDaemonProcess test_optimization_group user=apache group=apache processes=1 threads=5 request-timeout=200

# Might stop multithreading

    WSGIApplicationGroup %{GLOBAL}

    WSGIScriptAlias /wsgi "C:/Users/mysuser/PycharmProjects/test_optimization/test.wsgi"

    DocumentRoot "C:/Users/myuser/PycharmProjects/test_optimization"

</VirtualHost>

3. It is working as expected. Everything is fine till now.

4. Now the problem comes here when I add these two below imports from rp2 package:

     import rpy2.robjects as robjects

     from rpy2.robjects import panda2sri

The moment I add these above imports in my test.wsgi script Apache server stops responding rather it hangs without any error.
However the same piece of code (the two rpy2 package imports) is working as expected in terminal.

I'm clueless at this point to how to fix this. I needed help on this ??

5. However I did some googling and found this url : https://bitbucket.org/rpy2/rpy2/issues/136/robjectsr-function-hangs-with-mod_wsgi

    which is pretty close to what I have been looking for, on windows machine:
    In here the last three comments where Laurent Gautier is saying that 'R cannot do multithreading and using web servers on the Python side that use are not going to work well'

    To try this fix, I tried adding WSGIProcessGroup & WSGIDaemonProcess to bring down the processes to 1 and etc.

   WSGIProcessGroup test_optimization_group

   WSGIDaemonProcess test_optimization_group user=apache group=apache processes=1 threads=5 request-timeout=200

   But irony of the situation is WSGIDaemonProcess is not supported on windows.

   As given here in this link : 

   https://modwsgi.readthedocs.io/en/develop/configuration-directives/WSGIDaemonProcess.html

    Note that the WSGIDaemonProcess directive and corresponding features are not available on Windows.(from the official website)

6. I'm not sure whether fixing point 5 (above mentioned) would fix my issue or is there any fix is required for this.

   Please let me know if I have missed anything. Any help on this is most welcome. 

   Thanks for your help in advance.

[Graham Dumpleton]

On Windows, Apache relies on multithreading, because it can't use multiple processes. The only solution would be to bump the Apache configuration down to single thread, but then because it is only a single process, the result is impractical, as you can only handle one request at a time. I'd suggest using rpy2 with mod_wsgi on Windows as a result probably isn't going to work very well.

That said, but it would be hard to setup, you could run a background thread to do the rpy2 processing, and have request handlers use a queue between threads to deliver the task to the rpy2 specific thread and then wait for a response.

This isn't that simple though, as you need to ensure that only the code run by that special thread imports rpy2. Further, any data structures going back and forth in the queues can't be rpy2 data structures, you would need to translate everything to and from rpy2 data structures to native Python data structures, in the rpy2 thread. Lastly being a single processing thread, it could itself still become a bottle neck.

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/913cb743-0461-4ba3-9e0a-b235d7ffff47%40googlegroups.com.

[*Ali...]

Hello Graham,

So what would be the better solution for this, 

should I go for another server instead Apache or should I use Mac laptop instead?

Could you please suggest?

Thank you so much once again for you help.

Mohammad Nasim Ali.

To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/3B882014-5EA8-4129-BFA4-9B8D3510A7B3%40gmail.com.

[Graham Dumpleton]

Linux, or Linux in a container. Windows does not provide a good experience if need to do multi process Python applications.

To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/CAMfUNFLDRfbjP4TWe0NB2%3DkOWGZkOUeiWprEupGE5knZmnzAqQ%40mail.gmail.com.

[*Ali...]

Hello Graham,

Thank you so much for your help.

To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/A5AF835E-82AD-47D4-AB82-27997AFCD0DA%40gmail.com.