Need support for web2py application with over 1 million users.

[Sandeep]

We run major application in India with over 1 million users, and relatively high concurrent users traffic, which spikes twice a day. Recently we are experiencing a lot of troubles in scaling. We would love to get opinion from any consultant who provides support for web2py application in India. Any help in this regard would be appreciated. 

[Vinicius Assef]

Concerning web2py itself, the major scaling problem could be running big models files on each request. Using lazy tables will certainly help you.

After all, scaling is not just a framework issue, right? How about caching, load balancing, database indexes, etc.?

On 18 Apr 2015, at 15:41, Sandeep <shete....@gmail.com> wrote:

We run major application in India with over 1 million users, and relatively high concurrent users traffic, which spikes twice a day. Recently we are experiencing a lot of troubles in scaling. We would love to get opinion from any consultant who provides support for web2py application in India. Any help in this regard would be appreciated. 

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Massimo Di Pierro]

For use to help you we need more info about the architecture. There two ways you can do this.

1) you can share information on this list and we will all help and everybody will learn. 2) you can ask for private consulting from one of the companies that provide web2py support. If you need advice, please contact me personally.

Anyway. Web2py is designed to scale horizontally by adding server behind a load balancer. You can increase the number of requests per machine by improving caching. Eventually you hit the database bottleneck. That is a framework independent problem. If that is the problem you want to look into optimizing queries. Some times it can be done and some times it cannot. Some times you have to move to a NoSQL database like Google Cloud Storage or Mongo. 

Massimo

[Ramkrishan Bhatt]

GAE is better for scale application but developer should know indexing and optimized DAL Table. I guess you may facing problem while playing DAL objects in function.

[Michele Comitini]

Hi Sandeep,

To help us try help you, please let us know:

1. database

2. http web server

3. WSGI adapter (eg. uwsgi, fcgi etc..)

4. WSGI execution model (threading, processing)

5. OS

OR if you are running in some PaaS, which one.

2015-04-20 5:25 GMT+02:00 Ramkrishan Bhatt <ramkrish...@gmail.com>:

GAE is better for scale application but developer should know indexing and optimized DAL Table. I guess you may facing problem while playing DAL objects in function.

[Derek]

Barring him sharing that information, what general advice can you give regarding this? 

For example, should you store sessions in redis and setup squid and nginx on the hosts? How do you handle the migrations, etc?

[Paolo Valleri]

Mind that if you store session in the database/redis you could have problems in pages that use components (es: load). Otherwise, it is a good starting point.

[Michele Comitini]

There is no general advice...

Think you were a doctor receiving a call from an unknown person: "I feel bad, can you help me?"

What would you reply based on that information? "Take some vitamins" ??

--

[Niphlod]

Database, Yes. Redis, Nope with lock_Keys. I'd go as far as pointing out that the contrib redis backend is the fastest, less cumbersome and less maintenance needy of the whole set of possibilities.

[JorgeH]

It would be great to have a blog article written about this topic by the ones who know the most.

Or to update the web2py manual book with all these pointers.

[Vinicius Assef]

> On 24 Apr 2015, at 09:31, Michele Comitini <michele....@gmail.com> wrote:
>
> There is no general advice…

Yes, there are many, as some were already pointed out in this thread, others not yet:
- optimize DB queries;
- create indexes in your BD;
- session.forget() when not necessary;
- avoid big models scripts;
- caching is really useful;
- lazy tables, according to the book, has a big performance improvement;
- etc.

But, how about listening from the thread creator about questions we asked him?

—
Vinicius