Having gunicorn deploy the app with django?

[Mike Chung]

I am running into a few issues using gevent-socket.io and deploying with nginx gunicorn and django.

my wsgi.py file that I have gunicorn run looks like this.

------------------------------------------------------------------------------------------------

"""

WSGI config for QWebAppProj project.

It exposes the WSGI callable as a module-level variable named ``application``.

For more information on this file, see

https://docs.djangoproject.com/en/1.6/howto/deployment/wsgi/

"""

# Need to start the socket io server here.

from sys import stdout

from gevent import monkey

monkey.patch_all()

import os

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "QWebAppProj.settings")

from django.core.wsgi import get_wsgi_application

application = get_wsgi_application()

from socketio.server import SocketIOServer

server = SocketIOServer(

    ('', 8000), application, resource="socket.io", policy_server=False)

server.serve_forever()

------------------------------------------------------------------------------------------------------------------------

So I am using sdjango and autodiscovery in this url files here...

------------------------------------------------------------------------------------------------

QWebappProj.urls

from django.conf.urls import patterns, include, url

from django.contrib import admin

from qapp import urls

admin.autodiscover()

urlpatterns = patterns('',

                       # Examples:

                       # url(r'^$', 'QWebAppProj.views.home', name='home'),

                       # url(r'^blog/', include('blog.urls')),

                       url(r'^admin/', include(admin.site.urls)),

                       url(r'', include("qapp.urls"))

                       )

------------------------------------------------------------------------------------------------

Then in my qapp urls I have this

------------------------------------------------------------------------------------------------

from django.conf.urls import patterns, include, url

import socketio.sdjango

socketio.sdjango.autodiscover()

urlpatterns = patterns(

    # This is all that is needed to setup socket io it looks for an app that

    # contains socketio code.

    url("^socket\.io", include(socketio.sdjango.urls))

) 

------------------------------------------------------------------------------------------------

I am using socketio in this manner because I am using a native application to communicate with it.

I apologise for spamming your board, this is my last resort, I have been working on getting this stack up for days.

------------------------------------------------------------------------------------------------

I am getting this stack trace error which has to do with the way I setup the socket server.

[ERROR] Exception in worker process:

Traceback (most recent call last):

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/arbiter.py", line 495, in spawn_worker

    worker.init_process()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/workers/base.py", line 106, in init_process

    self.wsgi = self.app.wsgi()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/app/base.py", line 114, in wsgi

    self.callable = self.load()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 62, in load

    return self.load_wsgiapp()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load_wsgiapp

    return util.import_app(self.app_uri)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/util.py", line 354, in import_app

    __import__(module)

  File "/home/mikechung/Q/QWebAppProj/QWebAppProj/wsgi.py", line 26, in <module>

    server.serve_forever()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/baseserver.py", line 282, in serve_forever

    self.start()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/baseserver.py", line 234, in start

    self.init_socket()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/pywsgi.py", line 639, in init_socket

    StreamServer.init_socket(self)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/server.py", line 78, in init_socket

    self.socket = self.get_listener(self.address, self.backlog, self.family)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/server.py", line 89, in get_listener

    return _tcp_listener(address, backlog=backlog, reuse_addr=self.reuse_addr, family=family)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/server.py", line 153, in _tcp_listener

    sock.bind(address)

  File "<string>", line 1, in bind

error: [Errno 98] Address already in use: ('', 8000)

Traceback (most recent call last):

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/arbiter.py", line 495, in spawn_worker

    worker.init_process()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/workers/base.py", line 106, in init_process

    self.wsgi = self.app.wsgi()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/app/base.py", line 114, in wsgi

    self.callable = self.load()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 62, in load

    return self.load_wsgiapp()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load_wsgiapp

    return util.import_app(self.app_uri)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/util.py", line 354, in import_app

    __import__(module)

  File "/home/mikechung/Q/QWebAppProj/QWebAppProj/wsgi.py", line 26, in <module>

    server.serve_forever()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/baseserver.py", line 282, in serve_forever

    self.start()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/baseserver.py", line 234, in start

    self.init_socket()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/pywsgi.py", line 639, in init_socket

    StreamServer.init_socket(self)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/server.py", line 78, in init_socket

    self.socket = self.get_listener(self.address, self.backlog, self.family)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/server.py", line 89, in get_listener

    return _tcp_listener(address, backlog=backlog, reuse_addr=self.reuse_addr, family=family)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/server.py", line 153, in _tcp_listener

    sock.bind(address)

  File "<string>", line 1, in bind

error: [Errno 98] Address already in use: ('', 8000)

2014-03-20 17:37:44 [3849] [INFO] Worker exiting (pid: 3849)

2014-03-20 17:37:44 [3850] [ERROR] Exception in worker process:

Traceback (most recent call last):

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/arbiter.py", line 495, in spawn_worker

    worker.init_process()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/workers/base.py", line 106, in init_process

    self.wsgi = self.app.wsgi()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/app/base.py", line 114, in wsgi

    self.callable = self.load()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 62, in load

    return self.load_wsgiapp()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load_wsgiapp

    return util.import_app(self.app_uri)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/util.py", line 354, in import_app

    __import__(module)

  File "/home/mikechung/Q/QWebAppProj/QWebAppProj/wsgi.py", line 26, in <module>

    server.serve_forever()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/baseserver.py", line 282, in serve_forever

    self.start()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/baseserver.py", line 234, in start

    self.init_socket()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/pywsgi.py", line 639, in init_socket

    StreamServer.init_socket(self)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/server.py", line 78, in init_socket

    self.socket = self.get_listener(self.address, self.backlog, self.family)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/server.py", line 89, in get_listener

    return _tcp_listener(address, backlog=backlog, reuse_addr=self.reuse_addr, family=family)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/server.py", line 153, in _tcp_listener

    sock.bind(address)

  File "<string>", line 1, in bind

error: [Errno 98] Address already in use: ('', 8000)

Traceback (most recent call last):

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/arbiter.py", line 495, in spawn_worker

    worker.init_process()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/workers/base.py", line 106, in init_process

    self.wsgi = self.app.wsgi()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/app/base.py", line 114, in wsgi

    self.callable = self.load()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 62, in load

    return self.load_wsgiapp()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load_wsgiapp

    return util.import_app(self.app_uri)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gunicorn/util.py", line 354, in import_app

    __import__(module)

  File "/home/mikechung/Q/QWebAppProj/QWebAppProj/wsgi.py", line 26, in <module>

    server.serve_forever()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/baseserver.py", line 282, in serve_forever

    self.start()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/baseserver.py", line 234, in start

    self.init_socket()

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/pywsgi.py", line 639, in init_socket

    StreamServer.init_socket(self)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/server.py", line 78, in init_socket

    self.socket = self.get_listener(self.address, self.backlog, self.family)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/server.py", line 89, in get_listener

    return _tcp_listener(address, backlog=backlog, reuse_addr=self.reuse_addr, family=family)

  File "/home/mikechung/stackEnv/lib/python2.7/site-packages/gevent/server.py", line 153, in _tcp_listener

    sock.bind(address)

  File "<string>", line 1, in bind

error: [Errno 98] Address already in use: ('', 8000)

2014-03-20 17:37:44 [3850] [INFO] Worker exiting (pid: 3850)

^C2014-03-20 17:37:50 [3848] [INFO] Worker exiting (pid: 3848)

2014-03-20 17:37:50 [3837] [INFO] Shutting down: Master

2014-03-20 17:37:50 [3837] [INFO] Reason: Worker failed to boot.

------------------------------------------------------------------------------------------------

[Mike Chung]

I forgot to mention that I am having nginx create file socket that gunicorn subscribes to.

 #!/bin/bash

 

NAME="Q_app"                                  # Name of the application

DJANGODIR=/home/mikechung/Q/QWebAppProj             # Django project directory

SOCKFILE=/home/mikechung/Q/QWebAppProj/gunicorn.sock  # we will communicte using this unix socket

USER=mikechung                                        # the user to run as

GROUP=users                                    # the group to run as

NUM_WORKERS=3                                     # how many worker processes should Gunicorn spawn

DJANGO_SETTINGS_MODULE=QWebAppProj.settings             # which settings file should Django use

DJANGO_WSGI_MODULE=QWebAppProj.wsgi                     # WSGI module name

 

echo "Starting $NAME as `whoami`"

 

# Activate the virtual environment

cd /home/mikechung/stackEnv

source bin/activate

export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE

export PYTHONPATH=$DJANGODIR:$PYTHONPATH

 

# Create the run directory if it doesn't exist

RUNDIR=$(dirname $SOCKFILE)

test -d $RUNDIR || mkdir -p $RUNDIR

 

# Start your Django Unicorn

# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)

exec gunicorn ${DJANGO_WSGI_MODULE}:application \

  --worker-class socketio.sgunicorn.GeventSocketIOWorker \

  --name $NAME \

  --workers $NUM_WORKERS \

  --user=$USER --group=$GROUP \

  --log-level=debug \

  --bind=unix:$SOCKFILE

[Mike Chung]

Okay figured out the issue I was getting 502 because I was using more than 1 worker with gunicorn 18

[Daniel Swarbrick]

I've run into something similar, and found this patch to fix the problem:

https://github.com/arnuschky/gevent-socketio/commit/3cf45755e115816e342f6e3085dcf9fc89070ad8

Not sure if it's related, but this patch enabled us to finally run more than one gunicorn worker and have gevent-socketio work.