Cherokee server locking while upload
LightOfMooN
When I upload file, server is locking for all users.
Does somebody khow how to fix it?
Marin
Roberto De Ioris
> I have web2py + uWSGI + Cherokee
>
> When I upload file, server is locking for all users.
> Does somebody khow how to fix it?
>
You have to increase the number of workers (or threads).
Cherokee is a streaming server so it will pass POST data to uWSGI as
soon as they arrive leaving it blocked for all the upload cycle.
Add -p <n> (ncpu*2 is a good number to start with) or if you are using a
0.9.7.x release you can use threads with --threads <n>
--
Roberto De Ioris
http://unbit.it
LightOfMooN
But so if 4 users go to page with image, or upload some file, all will
be crashed again.
Is it a good to have <processes>1000</processes>?
Sounds bad
LightOfMooN
set <processes>10</processes>
but on the 4nd upload server locked
Roberto De Ioris
This is a normal deploy problem (not related to uWSGI). This is why
async/evented is so loved this days :) If your machine can tolerate 1000
processes (but sorry i do not think so) use this, but every process is a
full copy of your app stack so memory will be devoured.
If you have so many users doing uploads, you can try threads (they are
cheaper than processes) but the best solution would be Cherokee disabiling
streaming uploads to pass the request to uWSGI only when the upload is
done.
I will talk to the Cherokee lead developer to reintroduce this feature
(before 1 point releases Cherokee worked that way)
LightOfMooN
than 3k users online, which can upload and download files?
Roberto De Ioris
> Thx, but what is the best solution for now to deploy web2py with more
> than 3k users online, which can upload and download files?
No using a streaming upload webserver will solve your problem
Move to nginx, this will solve your specific problem, and use
X-Accel-Redirect (this is the equivalent of X-Sendfile in nginx) for all
the downloads.
>
> On 27 ���, 11:46, "Roberto De Ioris" <robe...@unbit.it> wrote:
>> > Thx, <processes>4</processes> helps.
>> > But so if 4 users go to page with image, or upload some file, all will
>> > be crashed again.
>> > Is it a good to have <processes>1000</processes>?
>> > Sounds bad
>>
>> This is a normal deploy problem (not related to uWSGI). This is why
>> async/evented is so loved this days :) If your machine can tolerate 1000
>> processes (but sorry i do not think so) use this, but every process is a
>> full copy of your app stack so memory will be devoured.
>>
>> If you have so many users doing uploads, you can try threads (they are
>> cheaper than processes) but the best solution would be Cherokee
>> disabiling
>> streaming uploads to pass the request to uWSGI only when the upload is
>> done.
>>
>> I will talk to the Cherokee lead developer to reintroduce this feature
>> (before 1 point releases Cherokee worked that way)
>>
>> --
>> Roberto De Iorishttp://unbit.it
>
LightOfMooN
Many parallel downloads, nice speed, and no locks even more.
Nginx Rocks!
Jose
On 27 mar, 06:54, LightOfMooN <vladsale...@yandex.ru> wrote:
> Yes, just tested it.
> Many parallel downloads, nice speed, and no locks even more.
> Nginx Rocks!
please, could show me how to configure nginx + + web2py uwsgi. I use
uwsgi + cherokee + web2py and I had the same problems that you
mentioned.
Thank you very much.
Jose
LightOfMooN
Thanks to pbreit for his script.
I just added support for 443 port (SSL) and Postgres to it.
It installs web2py+nginx+uwsgi+postgres on ubuntu:
===================
apt-get update
apt-get -y upgrade
apt-get -y install build-essential psmisc python-dev libxml2 libxml2-
dev python-setuptools
adduser --system --no-create-home --disabled-login --disabled-password
--group uwsgi
apt-get -y install unzip
cd /opt/
wget http://web2py.com/examples/static/web2py_src.zip
unzip web2py_src.zip
chown -R uwsgi:uwsgi web2py
cd web2py
sudo -u uwsgi python -c "from gluon.widget import console; console();"
sudo -u uwsgi python -c "from gluon.main import save_password;
save_password(raw_input('admin password: '),443)"
cd /opt/
wget http://projects.unbit.it/downloads/uwsgi-0.9.6.8.tar.gz
tar -zxvf uwsgi*
mv uwsgi*/ uwsgi/
cd uwsgi/
python setup.py install
chown -R uwsgi:uwsgi /opt/uwsgi
touch /var/log/uwsgi.log
chown uwsgi /var/log/uwsgi.log
apt-get -y install libpcre3-dev build-essential libssl-dev
cd /opt/
wget http://nginx.org/download/nginx-0.8.54.tar.gz
tar -zxvf nginx-0.8.54.tar.gz
cd /opt/nginx-0.8.54/
./configure --prefix=/opt/nginx --user=nginx --group=nginx --with-
http_ssl_module
make
make install
adduser --system --no-create-home --disabled-login --disabled-password
--group nginx
cp /opt/uwsgi/nginx/uwsgi_params /opt/nginx/conf/uwsgi_params
wget https://library.linode.com/web-servers/nginx/installation/reference/init-deb.sh
mv init-deb.sh /etc/init.d/nginx
chmod +x /etc/init.d/nginx
/usr/sbin/update-rc.d -f nginx defaults
/etc/init.d/nginx start
cd /opt/
wget https://library.linode.com/web-servers/nginx/python-uwsgi/reference/init-deb.sh
mv /opt/init-deb.sh /etc/init.d/uwsgi
chmod +x /etc/init.d/uwsgi
echo 'PYTHONPATH=/opt/web2py/
MODULE=wsgihandler' > /etc/default/uwsgi
/usr/sbin/update-rc.d -f uwsgi defaults
/etc/init.d/uwsgi start
echo 'user uwsgi;
worker_processes 4;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
client_max_body_size 100M;
server {
listen 80;
server_name "";
location / {
uwsgi_pass 127.0.0.1:9001;
include uwsgi_params;
}
location /static {
root /opt/web2py/applications/welcome/;
}
}
server {
listen 443;
server_name "";
ssl on;
ssl_certificate /opt/nginx/conf/server.crt;
ssl_certificate_key /opt/nginx/conf/server.key;
location / {
uwsgi_pass 127.0.0.1:9001;
include uwsgi_params;
uwsgi_param UWSGI_SCHEME $scheme;
}
location /static {
root /opt/web2py/applications/welcome/;
}
}
}' > /opt/nginx/conf/nginx.conf
/etc/init.d/nginx restart
# get sertificates
cd /opt/nginx/conf; openssl genrsa -out server.key 1024
cd /opt/nginx/conf; openssl req -batch -new -key server.key -out
server.csr
cd /opt/nginx/conf;
openssl x509 -req -days 1780 -in server.csr -signkey server.key -out
server.crt
/etc/init.d/nginx restart
#install PostgreSQL
sudo apt-get -y install postgresql
sudo apt-get -y install python-psycopg2
===================
take care with copy and paste it, because google break links and
newlines
pbreit
Jose
Thank you very much.