Minimum memory requirements?

[Alec Taylor]

I've just bought an account with WebFaction which gives me: 256MB memory, 100GB hard-drive space and 600GB bandwidth. (Apache)

I am hosting my web2py site (on latest trunk) and one blog on WordPress with this account.

When I changed the routes.py and then clicked "reload routes" it made my memory go over the limit to 320MB, which is why I ask.

What are the memory requirements?

Thanks for all information,

Alec Taylor

[Massimo Di Pierro]

Are you saying reloading routes, causes memory go up? What if you restart the web server? What is your configuration? what web2py and python version?

[Vasile Ermicioi]

I don't recommend using web2py with apache on webfaction, use uwsgi instead of apache (without nginx)

https://groups.google.com/forum/?fromgroups#!searchin/web2py/uwsgi$20webfaction/web2py/PWpwayGa4Io/bBVyehyZ3ogJ

[Alec Taylor]

Okay, will try that now. Looks promising

On Wed, Jul 4, 2012 at 12:28 AM, Massimo Di Pierro
<massimo....@gmail.com> wrote:
> Are you saying reloading routes, causes memory go up? What if you restart
> the web server? What is your configuration? what web2py and python version?
>

Running on Apache/2.2.17 (Unix) mod_wsgi/3.3 Python/2.7.3

[Alec Taylor]

Version 2.0.0 (2012-07-01 22:43:58) dev

[Alec Taylor]

Alright, I wrote this up as a guide (to help others in same
situation), so tell me if it's right or wrong:

Go to https://my.webfaction.com/applications and add this as an
application, keep note of port for below (replace "yourporthere" with
it):

$ curl http://projects.unbit.it/downloads/uwsgi-1.2.3.tar.gz >
uwsgi-1.2.3.tar.gz
$ tar xvzf uwsgi-1.2.3.tar.gz
$ cd uwsgi-1.2.3
$ python2.7 uwsgiconfig.py --build
$ mv uwsgi ~/bin
$ cd ..
$ rm -r uwsgi-1.2.3 uwsgi-1.2.3.tar.gz
$ wget <my private bitbucket URL with latest trunk in a .zip>
$ unzip web2py_src.zip
$ python2.7 web2py/web2py.py
$ mkdir ~/tmp
$ uwsgi --protocol=http --socket 127.0.0.1:yourporthere --pythonpath
~/web2py --module wsgihandler -d ~/tmp/uwsgi.log -t 20 --async 24
--ugreen --limit-as 64 -r --no-orphans -M -p 1 --touch-reload
~/tmp/uwsgireload.txt --reload-on-rss 50
$ nano ~/tmp/uwsgireload.txt
Paste this into it then save:
[utils]
ps -u fermer -o rss,command

[Vasile Ermicioi]

1) 

go to https://my.webfaction.com/applications and create an application, uwsgi for e.g.

App category: custom

App type: custom  app (listening on a port)

and once done you will see the port near app name (it is a number)

2) connect via ssh and execute those commands, but put instead yourportthere the number of port of your app

3)

[utils]

ps -u fermer -o rss,command

this is also a command, and shouldn't be written in uwsgireload.txt

fermer is my username on webfaction, you should put yours instead 

if you execute it, it will show you how much memory is used by your processes

this command shows you how much memory is using your processes

[Luther Goh Lu Feng]

At some point, will it be beneficial to have fabric scripts to setup web2py on webhosts?

http://fabfile.org

[Alec Taylor]

On Wed, Jul 4, 2012 at 2:27 AM, Vasile Ermicioi <elf...@gmail.com> wrote:
> 1)
> go to https://my.webfaction.com/applications and create an application,
> uwsgi for e.g.
> App category: custom
> App type: custom app (listening on a port)
> and once done you will see the port near app name (it is a number)
>
> 2) connect via ssh and execute those commands, but put instead yourportthere
> the number of port of your app
>
> 3)
> [utils]
> ps -u fermer -o rss,command
>
> this is also a command, and shouldn't be written in uwsgireload.txt
>
> fermer is my username on webfaction, you should put yours instead
> if you execute it, it will show you how much memory is used by your
> processes
>
>
>
> this command shows you how much memory is using your processes

Thanks, but it still doesn't seem to be working. Here is the output of
my ~/tmp/uwsgi.log:

*** Starting uWSGI 1.2.3 (64bit) on [Thu Jul 5 09:53:11 2012] ***
compiled with version: 4.4.6 20110731 (Red Hat 4.4.6-3) on 03 July 2012 09:59:58
detected number of CPU cores: 8
current working directory: /home/myusernamehere
detected binary path: /home/myusernamehere/bin/uwsgi
limiting address space of processes...
your process address space limit is 67108864 bytes (64 MB)
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow
upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 4096
async fd table size: 4096
allocated 24960 bytes (24 KB) for 24 cores per worker.
lock engine: pthread robust mutexes
initializing 24 uGreen threads with stack size of 262144 (256 KB)
mprotect(): Invalid argument [plugins/ugreen/ugreen.c line 109]

On Wed, Jul 4, 2012 at 6:22 AM, Luther Goh Lu Feng <elf...@yahoo.com> wrote:
> At some point, will it be beneficial to have fabric scripts to setup web2py
> on webhosts?
>
> http://fabfile.org
>

Sure, a fabric script would be helpful :)

[Alec Taylor]

Alright, fixed that last error, now it's just web2py (not uWSGI)
giving me trouble:

*** Operational MODE: async ***
added /home/myusername/web2py/ to pythonpath.
Traceback (most recent call last):
File "/home/myusername/web2py/wsgihandler.py", line 33, in <module>
import gluon.main
File "/home/myusername/web2py/gluon/__init__.py", line 15, in <module>
from globals import current
File "/home/myusername/web2py/gluon/globals.py", line 17, in <module>
from storage import Storage, List
File "/home/myusername/web2py/gluon/storage.py", line 15, in <module>
import cPickle
ImportError: PyCapsule_Import could not import module "cStringIO"
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***

[Vasile Ermicioi]

never saw such error, but multiple interpreter mode could mean you didn't installed uwsgi using specific python 

( in my case, I used python2.7)

wget http://projects.unbit.it/downloads/uwsgi-xx.tar.gz

tar xvzf uwsgi-xx.tar.gz

cd uwsgi-xx

/home/myusername/bin/python2.7 uwsgiconfig.py --build

mv uwsgi /home/myusername/bin

[Neil]

I'm following these instructions, and get stuck at the same point:

initializing 24 uGreen threads with stack size of 262144 (256 KB) 

mprotect(): Invalid argument [plugins/ugreen/ugreen.c line 109]  

How did you fix this?

[Neil]

Figured it out: I removed the "--limit-as 64" argument and now it seems to work. I'm using uwsgi version 1.2.4. Could this cause any problems?

Now, if I can get my static content served directly, I'm ready to roll with Webfaction. Previously, when using nginx+uwsgi, it was using 1-10MB of memory with every page view! I still don't really understand why this was not being released with that configuration. With the "--reload-on-rss" flag set, the uwsgi processes were being restarted every few minutes, and that was just with one person viewing the site (me)! 

[Alec Taylor]

I actually never got it to work on WebFaction...

I might try every week or so and spend 30 minutes on it, but frankly I
don't have the time.

Also, heroku and openshift are giving me issues with web2py, but it's
something before I throw web2py on it, which is why I haven't posted
on this mailing-list about it >.<

Lets just say I'm thoroughly annoyed I haven't been able to get web2py
hosted and working live

> --
>
>
>

[Massimo Di Pierro]

What is the main issue in your opinion? Did you try running a simple wsgi program without web2py?

[Alec Taylor]

When I currently visit the site it says:

"uWSGI Error

Python application not found"

The logfile says:

Python version: 2.7.3 (default, May 18 2012, 14:51:16) [GCC 4.4.6
20110731 (Red Hat 4.4.6-3)]
*** Python threads support is disabled. You can enable it with
--enable-threads ***
Python main interpreter initialized at 0x219d360
your server socket listen backlog is limited to 100 connections

*** Operational MODE: async ***

added /home/alectaylor/web2py/ to pythonpath.

Traceback (most recent call last):

File "/home/alectaylor/web2py/wsgihandler.py", line 33, in <module>
import gluon.main
File "/home/alectaylor/web2py/gluon/__init__.py", line 15, in <module>
from globals import current
File "/home/alectaylor/web2py/gluon/globals.py", line 17, in <module>

from storage import Storage, List

File "/home/alectaylor/web2py/gluon/storage.py", line 15, in <module>

import cPickle
ImportError: PyCapsule_Import could not import module "cStringIO"
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***

spawned uWSGI master process (pid: 1809)
spawned uWSGI worker 1 (pid: 1810, cores: 24)

> --
>
>
>

[Neil]

I came across that same error when I tried:

uwsgi --protocol=http --socket 127.0.0.1:yourporthere --pythonpath 
~/web2py --module wsgihandler -d ~/tmp/uwsgi.log -t 20 --async 24 

--limit-as 64 -r --no-orphans -M -p 1 --touch-reload 
~/tmp/uwsgireload.txt --reload-on-rss 50  

(NOTE: ugreen not included). However, when I put ugreen back in and removed "--limit-as 64", it worked (with ver 1.2.4)

Neil

On Sat, Jul 21, 2012 at 7:56 PM, :

> What is the main issue in your opinion? Did you try running a simple wsgi
> program without web2py?
>
>
> On Saturday, 21 July 2012 04:43:35 UTC-5, Alec Taylor wrote:
>>
>> I actually never got it to work on WebFaction...
>>
>> I might try every week or so and spend 30 minutes on it, but frankly I
>> don't have the time.
>>
>> Also, heroku and openshift are giving me issues with web2py, but it's
>> something before I throw web2py on it, which is why I haven't posted
>> on this mailing-list about it >.<
>>
>> Lets just say I'm thoroughly annoyed I haven't been able to get web2py
>> hosted and working live
>>

>> On Sat, Jul 21, 2012 at 7:15 PM,:

[Alec Taylor]

WOOOOOOOT

That worked, thanks a heap :D

> --
>  
>  
>  

[Massimo Di Pierro]

Neil, do you think it is work including a start-up script in web2py/scripts/? Would you submit one?

[Neil]

Sure, I'll create a script once I have a more confidence that everything is working. I still don't completely understand some of the settings I've found to work, so I'm a little hesitant...