Why web2py server hangs/freeze?

[Narendra Kumar]

Hi,

I stuck on the problem, as web2py server freezes/hang frequently. Almost daily.
I tried to track the reason, but I did not get any possible reason. Because I think this happen due to large amount of data processing and create file with large data or try to download big files. But later I found that It freezes/hangs any time.

Please help me.

Regards
Narendra Kumar

[Kiran Subbaraman]

It would help if you provide details of your setup
- OS version, python version, web2py version.
- configuration - do you use web2py with a webserver, and which one.
- do you see anything printed in the console, or web2py log file, or webserver log file when this happens?

Are you able to recreate this problem with any application that is running in web2py?
Also, this may help you debug "python processes that hang": https://wiki.python.org/moin/DebuggingWithGdb

________________________________________
Kiran Subbaraman
http://subbaraman.wordpress.com/about/

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Narendra Kumar]

I did not see any error or any warning in httpserver.log file as well as application's log.

Server configuration:

Linux OS = Oracle Linux Server release 6.8
Python =  2.6.6 (r266:84292, Jul 23 2015, 05:13:40) [GCC 4.4.7 20120313 (Red Hat 4.4.7-16)]
Web2py = 2.14.6-stable+timestamp.2016.05.09.19.18.48 (Running on Rocket 1.2.6, Python 2.6.6)

Run web2py by:
python web2py.py -a <mypassword> -c 138c3b68d25b8a7d.crt -k keyfile.key --ca-cert gd_bundle-g2-g1.crt -i 192.168.200.155 -p 443

Thanks
Narendra Kumar

[Kiran Subbaraman]

Am not sure what is causing the issue. The debug steps mentioned earlier will help you understand what is going on.

I understand you are on 2.14.x.
As an FYI, web2py 2.15.x on wards has deprecated support for python 2.6.
Search for 2.6 mentions on this page: https://github.com/web2py/web2py/blob/master/CHANGELOG

________________________________________
Kiran Subbaraman
http://subbaraman.wordpress.com/about/

--

[Massimo Di Pierro]

Let's rule out the usual culprits:

- when you saying it hangs are your trying from a new incognito window or from the same browser? It may be a session lock not a freeze.

- are using apache? Don't it has memory issues. Use Nginx

- do you have processes that may take a lot of time? consider adding session.forget() at the beginning of those actions and/or move the computation to a background task

- are you using sqlite? It does serialize all writes and locks the DB in the process. Have you tried postgresql?

- if you are using postgresql, have you checked the database logs? 

Massimo

[黄祥]

is put session.forget() at the beginning of the function can make process goes faster?

and
move the computation to a background task is same like put it on web2py scheduler?

thanks and best regards,

stifan

[Narendra Kumar]

Hi Massimo,

Good Morning. And sorry for my poor English.

Q1) when you saying it hangs are your trying from a new incognito window or from the same browser? It may be a session lock not a freeze.
Ans) Whenever sever freezes/hanged. We try to access our pages from different browser bit still get no response. At that time server never accept any request. The only option is to restart server.

Q2) are using apache? Don't it has memory issues. Use Nginx
Ans) As I already explained that I am running web2py using python command as mentioned below:. Not usng ang any server as apache/Nginx, etc.
        python web2py.py -a <mypassword> -c 138c3b68d25b8a7d.crt -k keyfile.key --ca-cert gd_bundle-g2-g1.crt -i <IP> -p <port>
       
Q3) do you have processes that may take a lot of time? consider adding session.forget() at the beginning of those actions and/or move the computation to a background task
Ans) I am not using session.forget(), as I maintain session expire after 20min. But 20 min is enough for any process, which perform heavy task.

Q4) are you using sqlite? It does serialize all writes and locks the DB in the process. Have you tried postgresql?
Q5) if you are using postgresql, have you checked the database logs?
Ans 4 & 5) I am using mysql database server only. Initially I think that large data process might create problem, but same table with same data works fine with PHP and JSF/Icefaces application, server takes time to process, but never be down like web2py.



Please let me know, I am running web2py directly using python is the problem, if it is, let me know the best server either apache/Nginx that can I use?.
Or Can increase memory used by python at runtime?.

Please help me.

Regards
Narendra Kumar

On Monday, September 4, 2017 at 7:17:25 PM UTC+5:30, Narendra Kumar wrote:

[Dave S]

On Sunday, September 10, 2017 at 10:44:25 PM UTC-7, Narendra Kumar wrote:

Hi Massimo,

Good Morning. And sorry for my poor English.

We can adjust; we have people posting from all over (but some of the people who learned English later can speak
better English than some of us native speakers).
 

Q1) when you saying it hangs are your trying from a new incognito window or from the same browser? It may be a session lock not a freeze.
Ans) Whenever sever freezes/hanged. We try to access our pages from different browser bit still get no response. At that time server never accept any request. The only option is to restart server.

Q2) are using apache? Don't it has memory issues. Use Nginx
Ans) As I already explained that I am running web2py using python command as mentioned below:. Not usng ang any server as apache/Nginx, etc.
        python web2py.py -a <mypassword> -c 138c3b68d25b8a7d.crt -k keyfile.key --ca-cert gd_bundle-g2-g1.crt -i <IP> -p <port>
       

[...]

 

Please let me know, I am running web2py directly using python is the problem, if it is, let me know the best server either apache/Nginx that can I use?.

An additional question is what OS are you hosting your webserver on, and then is it running server hardware, a virtual machine (such as a cloud instance), or a laptop?

But I think the core of the issue is that you aren't using NginX.  The invocation you give uses the Rocket server packaged with web2py, which is fine for development, but
   * has limited ability to scale for larger traffic loads
   * does seem to have some uptime issues

My experience with Rocket locking up happens on AWS linux, somewhere in the cloud. There are both short lockups (about 10 minutes) which are related to a socket error that times out, and indefinite lockups (hours) which I haven't yet associated with a specific problem.  The latter requires a kill the pid, restart the server cycle; at least, I don't have a better fix yet.

If you don't have a specific reason to go with Apache, go with NginX instead.  For both, you need to use uwsgi.  Many people deploy with NginX with just the setup script shipped with Web2py; be aware that if you have real certificates you have to tweak the script to not do the self-signed ones, and to point to the right place for the certs.  I am mulling over how to generalize this (I use Let's Encrypt), but haven't submitted anything yet.

I'm currently taking baby steps at moving to A Real Front End (because I have the luxury of being able to deep-dive on this).
For reading material, I found good NginX stuff (I think) on Pakt Pub, though it's not free.
<URL:https://www.packtpub.com/mapt/book/networking_and_servers/9781782165125/1/ch01lvl1sec04/installation>
<URL:https://www.packtpub.com/mapt/book/networking_and_servers/9781785289538/1/ch01lvl1sec09/configuring-nginx>

Free documentation on uswgi is pretty good:
<URL:https://uwsgi-docs.readthedocs.io/en/latest/WSGIquickstart.html>

Hope this helps.

Dave S
/dps

[Narendra Kumar]

Hi,

As suggested in previous posts, I deploy my web2py application on apache sever. I use scripts/setup-web2py-debian-sid.sh file to make setup on local server, which exists in web2py folder and follow steps defined on URL http://web2py.com/books/default/chapter/29/13/deployment-recipes#One-step-production-deployment.
But Still I get the server down issue.


Regards
Narendra

On Monday, September 4, 2017 at 7:17:25 PM UTC+5:30, Narendra Kumar wrote:

[Dave S]

On Thursday, September 14, 2017 at 10:40:46 PM UTC-7, Narendra Kumar wrote:

Hi,

As suggested in previous posts, I deploy my web2py application on apache sever. I use scripts/setup-web2py-debian-sid.sh file to make setup on local server, which exists in web2py folder and follow steps defined on URL http://web2py.com/books/default/chapter/29/13/deployment-recipes#One-step-production-deployment.
But Still I get the server down issue.


Regards
Narendra

If you're using Apache, then you should be looking in the Apache log files to see what it thinks happened.  If you find something in the log files that you don't understand, or that you think points to a bug in web2py, post that information here and we'll try to help.  (Well, I won't, because I'm going the nginx route, not having any existing Apache legacy to  live with, but there are several experience Apache users in the group.)

/dps

[Dave S]

On Monday, September 11, 2017 at 12:33:38 PM UTC-7, Dave S wrote:

 

My experience with Rocket locking up happens on AWS linux, somewhere in the cloud. There are both short lockups (about 10 minutes) which are related to a socket error that times out, and indefinite lockups (hours) which I haven't yet associated with a specific problem.  The latter requires a kill the pid, restart the server cycle; at least, I don't have a better fix yet.

I've gone almost 2 weeks since the last lockup, but had one this morning.  While it was down, I had a chance to use curl to try connecting, results below.

 curl -k -vv -X GET https://myserver.com
* About to connect() to myserver.com port 443 (#0)
*   Trying 52.34.144.39... connected
* Connected to myserver.com (52.34.144.39) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* warning: ignoring value of ssl.verifyhost
* NSS error -5961 (PR_CONNECT_RESET_ERROR)
* Closing connection #0
* SSL connect error
curl: (35) SSL connect error

 /dps

[Massimo Di Pierro]

we stopped recommending and supporting apache long ago. The world has moved to nginx and so have we.

these problems are apache problems not web2py problems and that is why we stopped supporting it.

[Narendra Kumar]

Hi,

I try to create setup web2py-nginx, using script setup-web2py-nginx-uwsgi-on-centos.sh exists in web2py /script folder.
During setup Error occur and need to stop setup, Below are the few line of error:

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
gcc -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes  -I. -IInclude -I./Include  -fPIC -DPy_BUILD_CORE -o Objects/funcobject.o Objects/funcobject.c
In file included from Include/Python.h:125:0,
                 from Objects/funcobject.c:4:
Include/modsupport.h:27:1: warning: ‘PyArg_ParseTuple’ is an unrecognized format function type [-Wformat=]
 PyAPI_FUNC(int) PyArg_ParseTuple(PyObject *, const char *, ...) Py_FORMAT_PARSETUPLE(PyArg_ParseTuple, 2, 3);
 ^
gcc -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes  -I. -IInclude -I./Include  -fPIC -DPy_BUILD_CORE -o Objects/intobject.o Objects/intobject.c
In file included from Include/Python.h:125:0,
                 from Objects/intobject.c:4:
Include/modsupport.h:27:1: warning: ‘PyArg_ParseTuple’ is an unrecognized format function type [-Wformat=]
 PyAPI_FUNC(int) PyArg_ParseTuple(PyObject *, const char *, ...) Py_FORMAT_PARSETUPLE(PyArg_ParseTuple, 2, 3);
 ^
gcc -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes  -I. -IInclude -I./Include  -fPIC -DPy_BUILD_CORE -o Objects/iterobject.o Objects/iterobject.c
In file included from Include/Python.h:125:0,
                 from Objects/iterobject.c:3:
Include/modsupport.h:27:1: warning: ‘PyArg_ParseTuple’ is an unrecognized format function type [-Wformat=]
 PyAPI_FUNC(int) PyArg_ParseTuple(PyObject *, const char *, ...) Py_FORMAT_PARSETUPLE(PyArg_ParseTuple, 2, 3);
 ^
gcc -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes  -I. -IInclude -I./Include  -fPIC -DPy_BUILD_CORE -o Objects/listobject.o Objects/listobject.c
In file included from Include/Python.h:125:0,
                 from Objects/listobject.c:3:
Include/modsupport.h:27:1: warning: ‘PyArg_ParseTuple’ is an unrecognized format function type [-Wformat=]
 PyAPI_FUNC(int) PyArg_ParseTuple(PyObject *, const char *, ...) Py_FORMAT_PARSETUPLE(PyArg_ParseTuple, 2, 3);

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

I am using OS: Oracle Linux Server release 7.3
Python:Python 2.7.5 (default, Sep  5 2016, 02:30:38)
           [GCC 4.8.5 20150623 (Red Hat 4.8.5-9)]
And this server is empty, having no other setup/plug-in/application, which may create problem.

Please help.

Regards
Narendra

[Narendra Kumar]

setup Script included in web2py, is not working properly.
I get a nice post, which work perfect.(https://www.digitalocean.com/community/tutorials/how-to-deploy-web2py-python-applications-with-uwsgi-and-nginx-on-centos-7)
By using Nginx, web2py working fine from last 4 weeks.

Regards
Narendra

[Alfonso Serra]

I have had the exact same problem. After a couple of requests the server hangs for a minute to get an undescriptive IOError.

I have been able to solve it by commenting these lines on one of my models.

from gluon.custom_import import track_changes
track_changes(True)

wasnt using modules anyway. Hope it helps.
Version 2.12.3-stable+timestamp.2015.08.19.00.18.03