Centos 5 script for Nginx with Uwsgi and Web2py

[peter]

Installing Web2py with Nginx and Uwsgi on Centos 5 is really tricky. There are lots of subtleties of ownership, and one has to take care when installing python 2.6 not to stop the systems python2.4 from working. I have finally developed a script that does all the installation from a clean start machine. The only thing that should need changing for another installation is the $basearch (base architecture) of the machine. This can be determined by doing 'uname -i'. This is needed for the nginx installation.

 

 

--------------------------------------------------------------------------------------------

 

# install development tools

yum install gcc gdbm-devel readline-devel ncurses-devel zlib-devel  bzip2-devel sqlite-devel db4-devel openssl-devel tk-devel bluez-libs-devel

 

#Install python 2.6 without overwriting python 2.4

#=================================

VERSION=2.6.8

mkdir ~/src

chmod 777 ~/src

cd ~/src

wget http://www.python.org/ftp/python/$VERSION/Python-$VERSION.tgz

tar xvfz Python-2.6.8.tgz

cd Python-2.6.8

./configure --prefix=/opt/python2.6 --with-threads --enable-shared

make

 

#The altinstall ensures that python2.4 is left okay

make altinstall

echo "/opt/python2.6/lib">/etc/ld.so.conf.d/opt-python2.6.conf

ldconfig

 

#create alias so that python 2.6 can be run with 'python2.6'

alias -p python2.6="/opt/python2.6/bin/python2.6"

ln -s /opt/python2.6/bin/python2.6 /usr/bin/python2.6

 

# Install uwsgi

#=========

version=uwsgi-1.2.3

cd /opt/

wget http://projects.unbit.it/downloads/$version.tar.gz

tar -zxvf $version.tar.gz

mv $version/ uwsgi/

cd uwsgi/

 

# build using python 2.6

python2.6 setup.py build

python2.6 uwsgiconfig.py --build

useradd uwsgi

 

#create and own uwsgi log

# Note this log will need emptying from time to time

echo " ">/var/log/uwsgi.log

chown uwsgi /var/log/uwsgi.log

 

#Install web2py

#==========

cd /opt

mkdir web-apps

cd web-apps

wget http://www.web2py.com/examples/static/web2py_src.zip

unzip web2py_src.zip

 

#set the ownership for web2py application to uwsgi

cd web2py

 chown -R uwsgi applications

chmod -R u+wx applications

 

#Now install nginx

#============

cd /etc/yum.repos.d

echo "[nginx]">nginx.repo

 

# in the line below replace 'i386' with the $basearch for your machine.

echo "baseurl=http://nginx.org/packages/centos/5/i386/">>nginx.repo

echo "gpgcheck=0">>nginx.repo

echo "enabled=1">>nginx.repo

yum install nginx

 

#We don't want the defaults, so remove them

cd /etc/nginx/conf.d

mv default.conf default.conf.o

mv example_ssl.conf example_ssl.conf.o

----------------------------------------------------------------------------------------------------------------

 

The following configuration files are also needed

 

The options for uwsgi are in the following file. It should be placed in /etc/uwsgi. Other options could be included.

 

uwsgi_for_nginx.conf

 

[uwsgi]

uuid=uwsgi

pythonpath = /opt/web-apps/web2py

module = wsgihandler

socket=127.0.0.1:9001

harakiri 60

harakiri-verbose

enable-threads

daemonize = /var/log/uwsgi.log

------------------------------------------------------------------------ 

The next configuration file is for nginx, and goes in /etc/nginx/conf.d It serves the static diretory of applications directly. I have not set up ssl because I access web2py admin by using ssh tunneling and the web2py rocket server. It should be straightforward to set up the ssl server however.

 

web2py.conf

 

server {

listen 80;

server_name $hostname;

location ~* /(\w+)/static/ {

root /opt/web-apps/web2py/applications/;

}

location / {

uwsgi_pass 127.0.0.1:9001;

include uwsgi_params;

}

}

#server {

#listen 443;

#server_name $hostname;

#ssl on;

#ssl_certificate /etc/nginx/ssl/web2py.crt;

#ssl_certificate_key /etc/nginx/ssl/web2py.key;

#location

#uwsgi_pass 127.0.0.1:9001;

#include uwsgi_params;

#uwsgi_param UWSGI_SCHEME $scheme;

#}

#}

 

 ------------------------------------------------------------------------

The final configuration file is only needed if you want to run uwsgi as a service. It should be placed in /etc/init.d

 

uwsgi_nginx

 

#!/bin/bash

 

# uwsgi - Use uwsgi to run python and wsgi web apps.

#

# chkconfig: - 85 15

# description: Use uwsgi to run python and wsgi web apps.

# processname: uwsgi

 

PATH=/opt/uwsgi:/sbin:/bin:/usr/sbin:/usr/bin

DAEMON=/opt/uwsgi/uwsgi

PYTHONPATH=/opt/web-apps/web2py

MODULE=wsgihandler

OWNER=uwsgi

 

NAME=uwsgi

DESC=uwsgi

 

test -x $DAEMON || exit 0

 

# Include uwsgi defaults if available

if [ -f /etc/default/uwsgi ] ; then

              . /etc/default/uwsgi

fi

 

set -e

 

get_pid() {

    if [ -f /var/run/$NAME.pid ]; then

        echo `cat /var/run/$NAME.pid`

    fi

}  

DAEMON_OPTS="--ini-paste /etc/uwsgi/uwsgi_for_nginx.conf"

 

case "$1" in

  start)

              echo -n "Starting $DESC: "

        PID=$(get_pid)

              echo "$PID"

        echo "B"

        if [ -z "$PID" ]; then

            echo "C"

            [ -f /var/run/$NAME.pid ] && rm -f /var/run/$NAME.pid

            echo "D"

            echo "$PID"

 

            touch /var/run/$NAME.pid                                        

            chown $OWNER /var/run/$NAME.pid

                  su - $OWNER -pc "$DAEMON $DAEMON_OPTS"

                  echo "$NAME."

                  echo "$PID"

        fi

 

              ;;

  stop)

              echo -n "Stopping $DESC: "

              pkill $NAME

#        kill -9 `ps -fu uwsgi | awk 'NR != 1 {print $2}'`

              ;;

  reload)

        echo "Reloading $NAME"

        PID=$(get_pid)

        [ ! -z "$PID" ] && kill -s 1 $PID &> /dev/null

        if [ $? -gt 0 ]; then

            echo "was not running"

            exit 1

        else

                  echo "$NAME."

            rm -f /var/run/$NAME.pid &> /dev/null

        fi

              ;;

  force-reload)

        echo "Reloading $NAME"

        PID=$(get_pid)

        [ ! -z "$PID" ] && kill -s 15 $PID &> /dev/null

        if [ $? -gt 0 ]; then

            echo "was not running"

            exit 1

        else

                  echo "$NAME."

            rm -f /var/run/$NAME.pid &> /dev/null

        fi

        ;;

  restart)

        $0 stop

        sleep 2

        $0 start

              ;;

  status) 

              killall -10 $DAEMON

              ;;

      *) 

                  N=/etc/init.d/$NAME

                  echo "Usage: $N {start|stop|restart|reload|force-reload|status}" >&2

                  exit 1

                  ;;

    esac

    exit 0

 

You can test it by

 

/etc/init.d/uwsgi_nginx start

 

and stop it similarly.

 

To add it as a service do

 

chkconfig --add uwsgi_nginx

chkconfig uwsgi_nginx o

You can test this with

 

service uwsgi_nginx start

 

nginx has automatically been set up as a service

if you now start it with

 

service nginx start

 

you should find the web2py welcome app will be displayed at your web address.

 

As they are both services, they should automatically start on a system reboot.

 

If you already had a server running, such as apache, you would need to stop that and turn its service off before running nginx.

 

 

 

 

 

 

 

 

 

 

---

No virus found in this message.
Checked by AVG - www.avg.com
Version: 2012.0.2177 / Virus Database: 2433/5067 - Release Date: 06/13/12

[Alan Etkin]

Hi peter, thanks for sharing the recipe. I followed most of the steps to deploy web2py with nginx and uwsgi in a CentOS ver 5.8 VPS host, but nginx is serving only the static default page. <host ip>/welcome or other url will fail with:

502 Bad Gateway

---

nginx/1.2.2

nginx was installed via yum, build uwsgi from source and stored it in etc/wsgi-python, and also installed Python 2.7.3 from source.
I had to change some locations in the config files to adapt to the actual server installations

Is there any shell or web server test I can perform to detect configuration issues?

[Alan Etkin]

Well, I finally found a way of getting the server to serve web2py apps. The only problem with your recipe for my host was in the startup script (maybe an error when doing copy/paste). I switched it with the one that comes with uwsgi source dist: centos_init_script and used your script's parameters. Here is the changed script:

#!/bin/bash

# uwsgi - Use uwsgi to run python and wsgi web apps.
#
# chkconfig: - 85 15
# description: Use uwsgi to run python and wsgi web apps.
# processname: uwsgi

# author: Roman Vasilyev

# Source function library.
. /etc/rc.d/init.d/functions

###########################
PATH=/etc/uwsgi-python:/sbin:/bin:/usr/sbin:/usr/bin
PYTHONPATH=/home/www-data/web2py
MODULE=wsgihandler
prog=/etc/uwsgi-python/uwsgi
OWNER=uwsgi
# OWNER=nginx ¿?
NAME=uwsgi
DESC=uwsgi
DAEMON_OPTS="-s 127.0.0.1:9001 -M 4 -t 30 -A 4 -p 16 -b 32768 -d /var/log/$NAME.log --pidfile /var/run/$NAME.pid --uid $OWNER --ini-paste /etc/uwsgi-python/uwsgi_for_nginx.conf"
##############################

[ -f /etc/sysconfig/uwsgi ] && . /etc/sysconfig/uwsgi

lockfile=/var/lock/subsys/uwsgi

start () {
  echo -n "Starting $DESC: "
  daemon $prog $DAEMON_OPTS
  retval=$?
  echo
  [ $retval -eq 0 ] && touch $lockfile
  return $retval
}

stop () {
  echo -n "Stopping $DESC: "
  killproc $prog
  retval=$?
  echo
  [ $retval -eq 0 ] && rm -f $lockfile
  return $retval
}

reload () {
  echo "Reloading $NAME"
  killproc $prog -HUP
  RETVAL=$?
  echo
}

force-reload () {
  echo "Reloading $NAME"
  killproc $prog -TERM
  RETVAL=$?
  echo
}

restart () {
    stop
    start
}

rh_status () {
  status $prog
}

rh_status_q() {
  rh_status >/dev/null 2>&1
}

case "$1" in
  start)
    rh_status_q && exit 0
    $1
    ;;
  stop)
    rh_status_q || exit 0
    $1
    ;;
  restart|force-reload)
    $1
    ;;
  reload)
    rh_status_q || exit 7
    $1
    ;;
  status)
    rh_status
    ;;
  *)  
    echo "Usage: $0 {start|stop|restart|reload|force-reload|status}" >&2
    exit 2
    ;;
  esac
  exit 0

Thanks

[Massimo Di Pierro]

Please email me a patch if this should go in scripts/

[peter]

Glad you got it working in the end. It is useful that it has now been independently tested.

Peter

[Alan Etkin]

The fixed peter scrip to install Python 2.7.3 and self-signed certs for ssl enabled apps. It retrieves the base architecture also for nginx installation. It should be tested although, as I only used its commands separately.

[Massimo Di Pierro]

In trunk. Kept the previous name. Added Peter's full name.

[peter]

Thank for extending the script Alan.

I have tested the trunk version. It fails when trying to start the service uwsgi

it cannot find /opt/uwsgi-python/uwsgi

This is presumably from

PROG=/opt/uwsgi-python/uwsgi

and 

  daemon $PROG $DAEMON_OPTS

Clearly when you tested it there was something else in the usgi_python folder. 

If you have a revised script I can test it.

Thanks

Peter

[Alan Etkin]

> El lunes, 30 de julio de 2012 10:20:16 UTC-3, peter escribió:Thank for extending the script Alan.

> I have tested the trunk version. It fails when trying to start the service uwsgi

> it cannot find /opt/uwsgi-python/uwsgi

Two things to check, I guess:

- Your system's /etc/sysconfig/uwsgi (if it is present) does not overwrite the default script variables
- The uwsgi installation does not fail to build, which would be the cause of not being available when starting the service

Note: according to uwsgi documentation there is no need of the line

python setup.py build

It is not executed in the example installation.

http://projects.unbit.it/uwsgi/wiki/Install

[peter]

 The instructions I gave at the top had been tested on a bare centos 5 machine and worked. 

I have got through 5 issues on your instructions so far:

'force-reload' not recognised so removed from uwsgi

 

'tar's had '-zxvf' as options, changed to 'zxvf'

 

'cp -R ./$version/* /opt/uwsgi-python/*'

changed to

'cp -R ./$version/* /opt/uwsgi-python'

after 

"cd /opt/uwsgi-python

echo 'build using python 2.7'

python2.7 setup.py build"

I get ImportError no module named setuptools.

Any ideas why you did not get this error?

Peter

On Sunday, 29 July 2012 15:26:37 UTC+1, Alan Etkin wrote:

[Alan Etkin]

I have got through 5 issues on your instructions so far:

'force-reload' not recognised so removed from uwsgi

Didn't test that command, it came with the example uwsgi startup script, but I think there's no need of it

'tar's had '-zxvf' as options, changed to 'zxvf'

tar man page examples use dash, perhaps it accepts both syntaxes.

'cp -R ./$version/* /opt/uwsgi-python/*'

changed to

'cp -R ./$version/* /opt/uwsgi-python'

The second argument of cp is not a valid location. My bad.
 

after 

"cd /opt/uwsgi-python

echo 'build using python 2.7'

python2.7 setup.py build"

I get ImportError no module named setuptools.

The setup script must be trying to import python libraries not installed, However, that command is not mandatory for building uwsgi. It should be enough to call

python2.7 uwsgiconfig.py --build

If that command builds uwsgi successfully under /opt/uwsgi-python, the script start command should work

[peter]

Attached is a working script that uses python2.6. It handles https. 

Python2.7 causes problems, eg no sqlite. This is why my original script was for python2.6. If anyone actually gets a Python2.7 script that they have tested on a bare centos 5 machine, then fine submit it. The attached script has been tested in this manner.

The only thing my script does not set up is the parameters_443.py file for the password. I copied this in manually.

Peter

[Alan Etkin]

> Python2.7 causes problems, eg no sqlite

I have web2py apps working both with sqlite and postgresql databases in the CentOS 5.8 host with Python 2.7.3

What other issues were detected? Would you please provide links or other info related to this problems? That could help improve the script also instead of simply discarding it.

[peter]

I did not just discard it, I spent a day working through the issues. I have a virtual box that I have installed Centos 5.8 on. I start with a clean install and apply the script.

With the changes I spoke of in my previous emails I got web2py starting but issuing a ticket. The error was a failure to import module. Googling this I found out it was sqlite3 not installed. I tried installing sqlite3, but python2/7 did not see it. At this point I decided to give up as I have no need for python2.7. Python2.6 is fine for me. I think that when I was developing the script I had the same problem and this is why I used python2.6

The script I attached in my previous email is your script but with python set to be 2.6. I tested this and it works. So we have a working script. If someone else wants to work on a script for python 2.7 this is fine by me. I think it saves a lot of people time if anyone submitting a script tests it on a bare operating system install. This is what I have done.

Peter

[Jonathan Lundell]

On 2 Aug 2012, at 6:32 AM, peter <peterchu...@gmail.com> wrote:

I did not just discard it, I spent a day working through the issues. I have a virtual box that I have installed Centos 5.8 on. I start with a clean install and apply the script.

With the changes I spoke of in my previous emails I got web2py starting but issuing a ticket. The error was a failure to import module. Googling this I found out it was sqlite3 not installed. I tried installing sqlite3, but python2/7 did not see it. At this point I decided to give up as I have no need for python2.7. Python2.6 is fine for me. I think that when I was developing the script I had the same problem and this is why I used python2.6

The script I attached in my previous email is your script but with python set to be 2.6. I tested this and it works. So we have a working script. If someone else wants to work on a script for python 2.7 this is fine by me. I think it saves a lot of people time if anyone submitting a script tests it on a bare operating system install. This is what I have done.

sqlite3 has been part of the Python standard library since 2.5. 

[Alan Etkin]

> The script I attached in my previous email is your script but with python set to be 2.6. I tested this and it works.

My concern is that 2.7 is said to be the last supported 2.x version of Python. I agree that testing in a raw install saves time and all kinds of issues. I will deploy in a virtual machine and debug the script so I can come back with a working one.

Shouldn't we provide the working script for the lastest CentOS distribution (which I supposse is not what we are using), instead of basing in a previous one?

[Massimo Di Pierro]

Do you suggest we replace the current script with this?

[peter]

Someone else has provided a script for centos 6. There are plenty of webservers out there based on centos 5 I believe because of its stability. So it is useful to have a script for Centos 5. Particularly as this seems to be the most tricky Linux for web2py because it uses and needs python 2.4

Massimo: it depends how long Alan will take to come up with a tested script for python 2.7. A working python2.6 script is better than an almost working 2.7 one.

Peter

[Alan Etkin]

Updated the script to:

- Write port 443 admin password
- Create welcome.w2p package (for solving admin create app feature error)
- Patch Python2.7.3 code for sqlite3 issue before installing

It was tested with a clean CentOS 5.8 vbox and is working. Needs improvement: it stops a few times for user confirmation

[peter]

Thanks for this Alan. I will independently test and feedback, but imagine it will work for me too.

Peter

[peter]

Script tested and working. Thanks Alan

Peter

[Massimo Di Pierro]

I have limited connectivity until tomorrow. If you have something to go in trunk please open an issue so it does not get lost.

[Christian Espinoza]

Hi Alan, I did test this script at a clean Centos 6, and the result is: 502 Bad Gateway

Christian

[Christian Espinoza]

I'm trying step to step now...and only was needed an little changes

[Alan Etkin]

I'm trying step to step now...and only was needed an little changes

Answered in web2py-usuarios

[peter]

I just got a new vps with centos 5 (32 bit on it). This script fails to get sqlite3 working.

As python 2.6 is fine I took the easy way and set

# Python options

PREFIX=2.6

VERSION=2.6.8

I also commented out the patch that is supposed to fix sqlite3 with python2.7

It works fine with these options. 

I think we would be wiser to make the standard script one for python 2.6 because of the problems with python 2.7

Peter

[Alan Etkin]

I just got a new vps with centos 5 (32 bit on it). This script fails to get sqlite3 working.

How? I thought I fixed it by applying the patch. IIRC the welcome app worked on 2.7 with a clean testing vm (and a sqlite connection).

Perhaps the scrip should set 2.6 by default and warn the user about the unstability of 2.7.x until they fix that in a new Python release. It would also need a conditional sentence to apply the patch in case the version is 2.7

[Alan Etkin]

> I think we would be wiser to make the standard script one for python 2.6 because of the problems with python 2.7

Just before opening an issue, what do you think? (see attached)

[peter]

Good solution Alan.

Peter

[Massimo Di Pierro]

In trunk!