Web2py in apache mod_wsgi with virtualenv

[Daniel Gonzalez]

Hi,

I am deploying a web2py application, and I will be serving it with Apache via mod_wsgi. I have the following WSGI directives in my virtual host configuration:

WSGIDaemonProcess web2py user=www-data group=www-data \
display-name=%{GROUP}
WSGIProcessGroup web2py
WSGIScriptAlias / /var/www/web2py/wsgihandler.py

This is working, but it is taking the system wide python installation. As a result, some packages are not found (since they are only present in my virtualenv). I would like to tell this particular virtual host (or the whole Apache, if there is no other way), to use the python installation in my virtual environment (/home/myuser/.virtualenvs/python2.7.2/bin).

Is it possible to configure this for Apache? Or better, just for my virtual host? I would like to cause as little effect as possible to the rest of the system (specifically, I do not want to modify the default python version used system wide)

I have tried different things, like:

Adding the following to wsgihandler.py:

activate_this = '/path/to/virtualenv/bin/activate_this.py'
execfile(activate_this, dict(__file__=activate_this)
import sys
sys.path.insert(0, '/path/to/web2py_dir')

Also, modififying the /etc/init.d/apache2 script, adding the following:

ENV="env -i LANG=C PATH=/home/myuser/.virtualenvs/python2.7.2/bin:/usr/local/bin:/usr/bin:/bin"
export PATH="/home/myuser/.virtualenvs/python2.7.2/bin:$PATH"
python -V

Strangely enough, the python -V shows that the right python binary is active (the one in the virtualenv, 2.7.2), but web2py, in the error reports, is showing the following:

web2py™ (2, 0, 0, datetime.datetime(2012, 7, 1, 22, 43, 58), 'dev')
Python Python 2.6.5: /home/myuser/.virtualenvs/python2.7.2/bin/python

So is it using 2.6.5 or 2.7.2? The python installed in /home/myuser/.virtualenvs/python2.7.2/bin/python *is* 2.7.2.

What I need is to use the virtualenv *and* python 2.7.2, which is not the system-wide python, but the one in the virtualenv.

Do you have any suggestions?

Thanks,

Daniel

[Jonathan Lundell]

On 1 Aug 2012, at 8:13 AM, Daniel Gonzalez <gonv...@gmail.com> wrote:

I am deploying a web2py application, and I will be serving it with Apache via mod_wsgi. I have the following WSGI directives in my virtual host configuration:

WSGIDaemonProcess web2py user=www-data group=www-data \
display-name=%{GROUP}
WSGIProcessGroup web2py
WSGIScriptAlias / /var/www/web2py/wsgihandler.py

This is working, but it is taking the system wide python installation. As a result, some packages are not found (since they are only present in my virtualenv). I would like to tell this particular virtual host (or the whole Apache, if there is no other way), to use the python installation in my virtual environment (/home/myuser/.virtualenvs/python2.7.2/bin).

Is it possible to configure this for Apache? Or better, just for my virtual host? I would like to cause as little effect as possible to the rest of the system (specifically, I do not want to modify the default python version used system wide)

http://code.google.com/p/modwsgi/wiki/VirtualEnvironments

[Daniel Gonzalez]

Quoting from the link you posted:

"The purpose of such Python virtual environments is to allow one to create multiple distinct Python environments for the same version of Python, but with different sets of Python modules and packages installed into the Python 'site-packages' directory."

This does not meet my requirements: I have different python versions. Actually, I have the system-wide python (2.6.5), which I *do not* want to use for mod_wsgi, and the python in my virtualenv (which I have bootrstrapped, version 2.7.2), which is the one that I want to use.

So, my virtualenv has both, packages and python binary. The reason is that my python (web2py) application needs a specific version of python, which is not available system wide.

So, how can I tell mod_wsgi to use the python binary in my virtualenv? Usually, just by setting the PATH to point to it (preceeding the path of the system wide python), all scripts starting with "#!/usr/bin/env python" would use the right python version, but this does not seem to work in case of mod_wsgi.

Any ideas?

Thanks,

Daniel

[Jonathan Lundell]

On 1 Aug 2012, at 8:52 AM, Daniel Gonzalez <gonv...@gmail.com> wrote:

Quoting from the link you posted:

"The purpose of such Python virtual environments is to allow one to create multiple distinct Python environments for the same version of Python, but with different sets of Python modules and packages installed into the Python 'site-packages' directory."

It (virtualenv) also supports different Python versions. Note later in the text where they talk about making sure that your mod_wsgi is built with the Python version you're using in the virtualenv.

[Daniel Gonzalez]

Thanks Jonathan,

On Wednesday, August 1, 2012 5:56:37 PM UTC+2, Jonathan Lundell wrote:

On 1 Aug 2012, at 8:52 AM, Daniel Gonzalez <gonv...@gmail.com> wrote:

Quoting from the link you posted:

"The purpose of such Python virtual environments is to allow one to create multiple distinct Python environments for the same version of Python, but with different sets of Python modules and packages installed into the Python 'site-packages' directory."

It (virtualenv) also supports different Python versions. Note later in the text where they talk about making sure that your mod_wsgi is built with the Python version you're using in the virtualenv.

I am not building mod_wsgi: I am installing it with apt-get install. I do not know which is the version it was compiled for, since it comes pre-packaged with the apache ubuntu distribution. I assume it is the system-wide python version (2.6.5).

But I need to use the python version in my virtualenv for my web2py application (2.7.2). Running *any* python script with *any* python interpreter is usually as easy as making sure that the right python interpreter is selected, by setting the PATH, and putting "#!/usr/bin/env python" on top of the python script. Why is mod_python different in this regard? Isn't it basically a wrapper to call the python interpreter?

Do you think there is any chance at all of using the python version in my virtualenv without having to recompile mod_wsgi?

I am afraid that if I start recompiling mod_wsgi this will prove very complicated, and it will even force me to recompile apache ...

Besides, what happens if tomorrow I need to upgrade my python version? Then I need to recompile mod_wsgi too. Looks complicated :(

Thanks,

Daniel

[Jonathan Lundell]

On 1 Aug 2012, at 9:30 AM, Daniel Gonzalez <gonv...@gmail.com> wrote:
> On Wednesday, August 1, 2012 5:56:37 PM UTC+2, Jonathan Lundell wrote:
> On 1 Aug 2012, at 8:52 AM, Daniel Gonzalez <gonv...@gmail.com> wrote:
>> Quoting from the link you posted:
>>
>> "The purpose of such Python virtual environments is to allow one to create multiple distinct Python environments for the same version of Python, but with different sets of Python modules and packages installed into the Python 'site-packages' directory."
>
> It (virtualenv) also supports different Python versions. Note later in the text where they talk about making sure that your mod_wsgi is built with the Python version you're using in the virtualenv.
>
> I am not building mod_wsgi: I am installing it with apt-get install. I do not know which is the version it was compiled for, since it comes pre-packaged with the apache ubuntu distribution. I assume it is the system-wide python version (2.6.5).
>
> But I need to use the python version in my virtualenv for my web2py application (2.7.2). Running *any* python script with *any* python interpreter is usually as easy as making sure that the right python interpreter is selected, by setting the PATH, and putting "#!/usr/bin/env python" on top of the python script. Why is mod_python different in this regard? Isn't it basically a wrapper to call the python interpreter?

I'm guessing because it's got some C components, and needs to be compiled against the same headers and libraries as the Python binary.

> Do you think there is any chance at all of using the python version in my virtualenv without having to recompile mod_wsgi?
>
> I am afraid that if I start recompiling mod_wsgi this will prove very complicated, and it will even force me to recompile apache ...
> Besides, what happens if tomorrow I need to upgrade my python version? Then I need to recompile mod_wsgi too. Looks complicated :(

I don't see why you'd need to recompile Apache, but yeah, it's a little complicated. Here's a discussion: http://code.google.com/p/modwsgi/wiki/InstallationIssues

Notice that you don't necessarily have to recompile mod_wsgi for a Python patch-level upgrade (third digit).

FWIW, I've done this both ways. Under Ubuntu, I built both Python and mod_wsgi to get 2.7.3. When I moved to RHEL, I decided that I could live with Python 2.6, and didn't bother with Python (though I did have to rebuild mod_wsgi for other reasons).

If you *do* build, I suggest using a tool like Fabric to encapsulate the process, so you can repeat it as required. Among other things, that makes building new servers or moving to later versions a lot easier. Here's my current fab script (note that it's oriented toward RHEL, but the Ubuntu details should be similar). The commented-out --with-python option lets you specify a non-system Python for the build.

@roles('web')
def install_wsgi():
if env.os == 'rhel':
e = dict(wsgi_version = '3.3')
with cd('/tmp'):
run('wget -nv http://modwsgi.googlecode.com/files/mod_wsgi-{wsgi_version}.tar.gz'.format(**e))
run('tar xvfz mod_wsgi-{wsgi_version}.tar.gz'.format(**e))
with cd('/tmp/mod_wsgi-{wsgi_version}'.format(**e)):
#./configure --with-python=/usr/local/bin/python
run('./configure')
run('make')
sudo('make install')
sudo('echo "LoadModule wsgi_module modules/mod_wsgi.so" > {apachedir}/conf.d/wsgi.conf'.format(**env))

[Daniel Gonzalez]

Thanks Jonathan,

I will try this later today.

Regards,

Daniel

[Daniel Gonzalez]

Thanks for your help Jonathan. I was able to get it working. I had to recompile the  python in my virtualenv too, with --enable-shared.

For reference, the related discussion in modwsgi: https://groups.google.com/d/topic/modwsgi/gQPT3iEwvU8/discussion