Internal Server Error 500 when doing medium to heavy data processing

[Davidiam]

Version

2.18.5-stable+timestamp.2019.04.08.04.22.03
(Running on Microsoft-IIS/10.0, Python 3.6.8)

Hello,

We have been using web2py for a couple of years with very few issues, but lately we have been getting Internal Server Error 500 error messages when certain scripts are called.  These scripts due some relatively heavy processing.

I am not sure if this is related to web2py or purely a server, resource or network problem but I thought that someone here may have had similar issues and can recommend some configuration change to remedy this.

Kind Regards, 

David

[Jirawat Kusolpreedee]

Try increasing timeout

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/7e70675f-7887-4b8a-964c-95e3a3d759c4n%40googlegroups.com.

[Dave S]

On Thursday, August 26, 2021 at 4:34:43 AM UTC-7 david....@gmail.com wrote:

Version

2.18.5-stable+timestamp.2019.04.08.04.22.03
(Running on Microsoft-IIS/10.0, Python 3.6.8)

Hello,

We have been using web2py for a couple of years with very few issues, but lately we have been getting Internal Server Error 500 error messages when certain scripts are called.  These scripts due some relatively heavy processing.

Are these scripts running outside of web2py or as modules within the application?  ("Scripts"  has so many connotations that it is a good idea to be more specific)

If the scripts are outside, can they be launched as independent processes?  If they are modules, can you use the scheduler to run them?   With either of those and a long-running processing job, I'd invoke the job from the page (link) that you use now, but instead of waiting for the results just return a link to a page with the results.  This gives the user quick feedback that the system has accepted the request, but the two-step process adds a little extra time without screwing with your timeouts, but as processing time increases you can have the 2nd link respond with "busy" until it isn't.  If you don't like the user to actually have to click that second link, you can experiment with the LOAD() apparatus, which also allows you to display a busy icon.

 

I am not sure if this is related to web2py or purely a server, resource or network problem but I thought that someone here may have had similar issues and can recommend some configuration change to remedy this.

Kind Regards, 

David

You haven't sketched a lot of context,   I've taken a stab at an answer based on my workflows, some of which were learned here 5 or  more years ago.  One place where I've used the above technique was in discovering certain types of work nodes, which involved sending queries over the network

I've done other processing where a file is chewed upon for certain keywords; with the link being used to upload the file for processing.  The application says "thank you, got it", schedules the processing, and returns HTTP 200.  The processing, when it completes, sends email telling the Interested Parties where the report is.

/dps