TG 1.5.1 + uWSGI with more than one process/worker fails

[Rama]

Hello people,

I still seem to be facing problems with my TG site even using uwsgi
(I was using TG's webd's before)

Currently, we are running the site with just a master process,
with this settings:

(venv)vodo@main:/srv/vodo-net/config$ cat uwsgi/uwsgi.ini
[uwsgi]
socket = /var/run/vodo/uwsgi.socket
pidfile=/var/run/vodo/uwsgi.pid
wsgi-file = /srv/vodo-net/vodo_uwsgi.py
daemonize = /var/log/vodo/uwsgi.log
enable-threads = true
chmod-socket = true
virtualenv = /srv/venv

If I try adding:

master = true
processes = 10

I get a 'commands out of sync' exception.

Full backtrace here:
http://dpaste.com/701086/

Additionally, if I run just one process for uwsgi, there seems to be a
leak somewhere because uwsgi gets to take up all memory
until the OOM finally kills it.

Could someone give any hints on how to debug this?
How could I do profiling to my app?

I must say I have no experience with profiling though, but could get
to learn it, of course.

Thanks again for all the help!
Rama

[Carlos Daniel Ruvalcaba Valenzuela]

You could try setting threading off unless you spaw threads on your tg
app, uwsgi does not make use of threads in your config and test if
that helps.

Also what version of uwsgi are you using? there seems to be problems
with 1.0.3 I think, it would be useful to cross post this to the uwsgi
list. Although this seems to be a db connectivity related problem too.

[Rama]

On 10 feb, 21:24, Carlos Daniel Ruvalcaba Valenzuela

Thanks for your tips Carlos,

I don't spaw threads on tg, so have disabled threads on uwsgi.

I was using version 1.0.2.1, and tried upgrading to latest stable
(1.0.4) but the same problem occurs.

( ProgrammingError: Commands out of sync; you can't run this command
now )

I have also tried upgrading SQLObject from 0.11.0 to latest (1.2.1)
but it didn't help either.

My uwsgi.ini looks like this:

[uwsgi]
socket = /var/run/vodo/uwsgi-staging.socket
pidfile=/var/run/vodo/uwsgi-staging.pid
wsgi-file = /srv/staging/vodo-net/vodo_uwsgi-staging.py
daemonize = /var/log/vodo/staging/uwsgi.log
enable-threads = false
chmod-socket = true
virtualenv = /srv/staging/staging-venv

master = true
processes = 10

I will post to uwsgi list now, but if anybody has any tips, all are
welcome.

Thanks again,
Rama

[Rama]

On 10 feb, 21:24, Carlos Daniel Ruvalcaba Valenzuela
<clsdan...@gmail.com> wrote:

One more thing, I have a concern as before the 'out of sync' error,
there is another exception showing up
which says 'SQLObjectNotFound: The object Visit by the ID 0 does not
exist'.

Full backtrace here:

http://dpaste.com/708425/

Could that have something to do?

Rama

[Rama]

Hello people, I've finally solved all my issues with help from the
uWSGI crew.

Just for the record: the solution to have multiple processes is to
enable lazy workers startup.

Paste from uwsgi reply:

---cut---
It is a bit of time i do not use SQLObject, but i bet it is not
copy-on-write frindly. Add

lazy = true

in your config.

In such a way, your app will be initialized after each worker spawn
(and
not before and then copied)
---cut---

Additionally, I've added:

max-requests = 3000

in order to recycle workers after uwsgi has served 3000 requests, as
it seems to consume memory without releasing it otherwise.

Everything looks to be working perfectly fine now.

Infinite thanks for all your help!
Rama

[Christoph Zwerschke]

Am 05.03.2012 14:36, schrieb Rama:
> Hello people, I've finally solved all my issues with help from the
> uWSGI crew.
>
> Just for the record: the solution to have multiple processes is to
> enable lazy workers startup.

Thanks for reporting back. I will add that in the docs.

-- Christoph