File upload/download over https freezes site

[Dave]

I have this same behavior on multiple web2py servers.  If a large file is being uploaded using a SQLFORM or downloaded using the default download controller, over HTTPS, the entire web server becomes unresponsive until the transfer is completed or cancelled.  However, I have no issues uploading/downloading the same file over HTTP, which can also take several minutes to complete, but the web server is still responsive during those transfers.

I am using the one-step deployment with Apache and a wildcard certificate (RapidSSL).  Would switching to nginx or cherokee give better performance for https file transfers, or is this likely an issue with the SSL certificate format?  Or if the file transfers over HTTPS are too CPU intensive, am i better off setting up multiple servers and a load balancer?

Thanks!
Dave R

[Niphlod]

uhm, you left out some pretty specific details.... what resources has the server web2py is deployed on ? moreover, what's the size of the file ? and what code are you using to handle the upload? are you using the default 'upload' Field or is it in conjunction with a 'blob' one to store the file on the database ?

[Dave]

Actually, it looks like i was chasing the wrong issue... It wasn't https after all.

Everything seems to be working after changing this line in apache default.conf:

WSGIDaemonProcess web2py user=www-data group=www-data processes=1 threads=1

to:

WSGIDaemonProcess web2py user=www-data group=www-data processes=5 threads=15

Is there any reason not to change this default setting from one-step deployment?  Can I likely set these values higher based on my hardware?

Thanks again,

Dave

[Dave]

Sorry about that, here are those hardware details:

Ubuntu server 14.04.01
CPU: Intel Xenon X5660 @ 2.80Ghz (dual core)
RAM: 8GB

[Niphlod]

I dropped off the apache train too soon to have any issues with it, but frankly, given the total sum of issues encountered so far on the forums, I'm starting to think that we'd need to "officially discontinue" our apache support.. may be total lack of luck in setting it up or very biased perspective, or total lack of internal knowledge but it seems that every problem that pops up with deployments have apache as the common ground.

looks like this is the commit to be blamed

https://github.com/web2py/web2py/commit/2a062a2ff5aa1e07e7bfcfdbf36b7f72e8aac5b4

I don't know the specifics around it but if it acts like it suggests, 1 thread and 1 process as a total sum aren't really worth of a production deployment.

[Dave]

That would be good to know.  Nothing in the web2py docs suggests that Apache should not be used in production, it actually seems more like the default since it is the first discussed in the documentation and it's used in the one-step production deployment.

The line of code in question is also in the one-step deployment script here:
http://web2py.googlecode.com/hg/scripts/setup-web2py-ubuntu.sh

[Derek]

looks like that change came from here:

https://groups.google.com/forum/#!searchin/web2py/processes$3D1$20threads$3D1|sort:date/web2py/jumZFKX2614/pu-NNXSMKHgJ

based on the post from Thomas...

Thomas J.

10/3/13

I've recently been comparing Web2py and PHP, this is what i found, maybe it helps:

1.: PHP is faster whereever it can do stuff within the interpreter, that includes handling post/get-data or templates. The PHP interpreter is written in C, so these things are really fast, whereas Web2py has to to them in Python.
2. DB access is heavily dependent on how many records you retrieve. Translating a query from DAL to SQL is basically free (so using the DAL syntax for DB access isn't an issue), putting the data into Python objects is quite expensive however. Only query what you really need. Also, using executesql() instead of the regular DAL syntax may help there, as it returns tuples, not complex data structures
3. I've found the default config for Apache2 to be somewhat broken as far as concurrency is concerned. Check your Apache/WSGI config, you probably have a line like "WSGIDaemonProcess web2py user=www-data group=www-data" in there. This actually defaults to 1 process with 15 threads, which -- on my machine -- completely kills performance (Python doesn't do threads well because of the global interpreter lock). I've found that even something like "WSGIDaemonProcess web2py user=www-data group=www-data processes=1 threads=1" improves performance for concurrent requests dramatically. Try tinkering with the processes-value to find the best config for your machine. 

You should probably change processes before you change threads.

[Dave]

Thanks for tracking that down!  So based on that post, and this one, it looks like i should be setting proccesses = # of cores, and threads = 1?

https://groups.google.com/forum/#!topic/web2py/mPdn1ClxLTI

Massimo DI Pierro:
There are pros and cons. If you use threads in a python program, the more computing cores you have, the slower - not faster - the program gets. This is a python feature because even if you have threads, there is only one interpreted and therefore execution is serialized anyway. For scalability you should have processes (not threads) one per core.

[Derek]

I believe that's what Massimo is trying to say. Each processor has it's own GIL.

[Niphlod]

let's not spread misinformations .... python has the GIL, true, and 15 threads with a single process won't use any cpu a single thread can use, nor span multiple cpus. But for anyone's sake, we're deploying a webapp in PRODUCTION, whose purpose should be - at the very least - be concurrent. 

If we force 1 process and a single thread, make ourselves a clap, than bang the head to the wall. we're serializing everything! 

tl;dr: I understand someone may think that 1 process with 15 threads (defaults of the apache directive) slow things down when you have 4xcpu and 3 of them remain underutilized while serving concurrently 15 requests (buh huh GIL, but I'd like to see test results), and that maybe a 4 processes with each 5 threads would be a smarter choice (really?!) but 1 process and 1 thread is a complete joke. 

[Dave]

I appreciate you sharing your knowledge on this!  Sounds like it may have been a bad commit?  Or, at least there should be some mention in web2py documentation about changing these values, mainly based on hardware?

Thanks again, sounds like my best bet is to reconfigure the server for quad core CPU, and set processes=4, and leave threads=15 (or lower, but not 1).  I realize my code needs to be "thread-safe", which I need to look into more, but it seems to work fine now with threads=15.

Dave

[Massimo Di Pierro]

I think that commit should be reversed. O changed to processes=5 threads=1. but this may cause an increased memory usage and it is not a good thing with apache. Agreed that Apache is no longer recommended (not just for web2py users) but we should continue to support it because it is still the mots popular web server.

[Dave]

Thanks for weighing in on this.  I'll update my apache config and keep an eye on memory usage.

Are there any plans for a one-step deployment script for a web server other than apache?  Which web server is the next most popular option with web2py community?

Dave

[Michele Comitini]

nginx coupled with uwsgi is quite popular and easy to maintain compared to apache.  scripts are in the scripts directory they also take care of installing uwsgi

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dave]

Thanks, I didn't realize there were all those different install scripts!  The web2py book doesn't mention them on the deployment recipes page, but i should probably keep a closer eye on github repository to see what's new.

I'm glad to see that apache works after changing these settings, and the change has already been committed for next web2py release, but I'll probably start transitioning to nginx going forward.

Dave

[Niphlod]

IMHO given that the majority of issues reported come with "one step production deployment" keywords, the book chapter http://web2py.com/books/default/chapter/29/13/deployment-recipes#One-step-production-deployment should be redacted to point to the script provisioning uwsgi and nginx instead of the bugged one provisioning apache.

[Massimo Di Pierro]

Yes. It should. I agree. We should leave the sections on Apache but comment they we do not recommend apache.

[Rafal Dobosz]

What are some servers you do recommend?

[Massimo Di Pierro]

We strongly recommend nginx for most users. Rocket (built-in), Tornado, and Unicorn all work great too but they do not have as many customizations as nginx.