Pylons/Pyramid Performance
Biswas, Pinakee
Hi,
As mentioned earlier, we are planning to use Pylons/Pyramid for our platform.
Could you please let me know if there is any performance statistics available for Pylons/Pyramid?
Is there any major portal with high traffic that is based on Pylons/Pyramid? I know one major Portal (with almost 12000 views per sec and few petabytes of monthly bandwidth usage) uses Python but not sure if it uses any web framework.
Looking forward to your response…
Thanks,
Pinakee Biswas
Director & CTO
Just watch it !
yGSM Mobile (Voice & Data): +91-982-016-7979
(Tel: +91-22-26733737 I (Fax:+91-22-26732828
7E- Mail: pin...@vvidiacom.com I 8Web: http://www.vvidiacom.com
P Please don't print this e-mail unless you really need to, this will preserve trees on planet earth.
----------------------------Disclaimer-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The information contained in this message (including any attachments) is confidential and may be privileged. If you have received it by mistake please notify the sender by return e-mail and permanently delete this message and any attachments from your system. Please note that e-mails are susceptible to change and malwares. VVIDIA COMMUNICATIONS PVT LTD. (including its group companies) shall not be liable for the improper or incomplete transmission of the information contained in this communication nor for any delay in its receipt or damage to your system.
-------------------------------------------------------------------------------------------------------------------------Disclaimer----------------------------------------------------------------------------------------------------------
chandrakant kumar
--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To post to this group, send email to pylons-...@googlegroups.com.
To unsubscribe from this group, send email to pylons-discus...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
--
regards
Andi
(sent right out of my head)
Hi,
As mentioned earlier, we are planning to use Pylons/Pyramid for our platform.
Could you please let me know if there is any performance statistics available for Pylons/Pyramid?
Is there any major portal with high traffic that is based on Pylons/Pyramid? I know one major Portal (with almost 12000 views per sec and few petabytes of monthly bandwidth usage) uses Python but not sure if it uses any web framework.
Looking forward to your response…
Thanks,
Pinakee Biswas
Director & CTO
<image001.jpg>
Just watch it !
yGSM Mobile (Voice & Data): +91-982-016-7979
(Tel: +91-22-26733737 I (Fax:+91-22-26732828
7E- Mail: pin...@vvidiacom.com I 8Web: http://www.vvidiacom.com
<image002.png>
P Please don't print this e-mail unless you really need to, this will preserve trees on planet earth.
----------------------------Disclaimer-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The information contained in this message (including any attachments) is confidential and may be privileged. If you have received it by mistake please notify the sender by return e-mail and permanently delete this message and any attachments from your system. Please note that e-mails are susceptible to change and malwares. VVIDIA COMMUNICATIONS PVT LTD. (including its group companies) shall not be liable for the improper or incomplete transmission of the information contained in this communication nor for any delay in its receipt or damage to your system.
-------------------------------------------------------------------------------------------------------------------------Disclaimer----------------------------------------------------------------------------------------------------------
--
Ronan Amicel
Is there any major portal with high traffic that is based on Pylons/Pyramid? I know one major Portal (with almost 12000 views per sec and few petabytes of monthly bandwidth usage) uses Python but not sure if it uses any web framework.
Vlad K.
> just a benchmark, but better than nothing. found that during our
> research.
> http://blog.curiasolutions.com/the-great-web-framework-shootout/
>
> andi
>
> (sent right out of my head)
>
synthetic benchmark it is highly unbalanced. For example, it shows
Pyramid yielding more rps than Bottle on Hello World. But then throws
Pyramid way lower than Bottle on templated db task.
If you take a look at the benchmark code, you'll notice:
- both use SQLite with a local file db, which is ok
- Bottle uses SQLite driver directly
- Pyramid uses SQLAlchemy which incurs significant overhead
You can use database drivers directly in Pyramid. You don't need
sqlalchemy, or transaction extensions, they are not required by Pyramid,
just a chosen default. You can also use SQLAlchemy in Bottle. You can
use whatever template engine, or none, in Pyramid, and Bottle.
The bottom line is Pyramid is not a full stack framework like Django,
and that means you can swap out default components for whatever you
want, just like you can add components to a microframework. The raw
speed of a Pyramid app then really depends on the choice of components
and app design. The framework overhead becomes insignificant.
That said, I'd totally trust Pyramid to power my next zettascale
Facebook killer. ;)
--
.oO V Oo.
Malthe Borch
> Could you please let me know if there is any performance statistics
> available for Pylons/Pyramid?
non-trivial application, then Python probably isn't fast enough.
Otherwise, Pyramid's performance depends mostly on how many requests
you can serve without expensive I/O (complex queries and/or disk I/O),
because the framework overhead is trivial.
\malthe
Biswas, Pinakee
Hi Vlad,
Thanks for the information and it definitely helps (also, would like to thank Andi for the statistics).
I have been primarily a C/C++ system level programmer and architected and developed large Telecom systems (carrier grade) requiring great performance levels. Currently looking into Python and corresponding web frameworks for our platform.
Some of the aspects of the design I have been pondering upon are also:
· How do I handle the threads, configure them etc. at optimum level? Do I really need to bother about them with Pylons/Pyramid? The platform is Linux.
· How is the performance of Paster Web server as compared to something like lighttpd (which is supposedly a light weight server)? If lighttpd, then I would need to embed Pylons/Pyramid there. (I have embedded Pylons with Apache but haven’t checked the performance).
· Regarding DB, SQLAlchemy can be a choice for SQL based DB servers. Considering something like MongoDB, not sure how would SQLAlchemy help there. But certainly if I have my own DB access library using native APIs, probably a better performance can be achieved.
If you or anyone has any thoughts on the above points, I would really appreciate.
Regards,
Pinakee
Please don't print this e-mail unless you really need to, this will preserve trees on planet earth.
----------------------------Disclaimer-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The information contained in this message (including any attachments) is confidential and may be privileged. If you have received it by mistake please notify the sender by return e-mail and permanently delete this message and any attachments from your system. Please note that e-mails are susceptible to change and malwares. VVIDIA COMMUNICATIONS PVT LTD. (including its group companies) shall not be liable for the improper or incomplete transmission of the information contained in this communication nor for any delay in its receipt or damage to your system.
-------------------------------------------------------------------------------------------------------------------------Disclaimer----------------------------------------------------------------------------------------------------------
--
Andreas Kaiser
Am 06.06.2012 um 13:23 schrieb Biswas, Pinakee:
> Some of the aspects of the design I have been pondering upon are also:
> · How do I handle the threads, configure them etc. at optimum level? Do I really need to bother about them with Pylons/Pyramid? The platform is Linux.
> · How is the performance of Paster Web server as compared to something like lighttpd (which is supposedly a light weight server)? If lighttpd, then I would need to embed Pylons/Pyramid there. (I have embedded Pylons with Apache but haven’t checked the performance).
> · Regarding DB, SQLAlchemy can be a choice for SQL based DB servers. Considering something like MongoDB, not sure how would SQLAlchemy help there. But certainly if I have my own DB access library using native APIs, probably a better performance can be achieved.
SQLAlchemy is a ORM and will therefore work only with Relational Databases.
HTH,
Andreas
Mike Orr
> On 06/06/2012 07:49 AM, Andi wrote:
>>
>> just a benchmark, but better than nothing. found that during our research.
>> http://blog.curiasolutions.com/the-great-web-framework-shootout/
>>
>> andi
>>
>> (sent right out of my head)
>>
>
>
> The Curiasolutions shootout is interesting. However, even for a synthetic
> benchmark it is highly unbalanced. For example, it shows Pyramid yielding
> more rps than Bottle on Hello World. But then throws Pyramid way lower than
> Bottle on templated db task.
>
> If you take a look at the benchmark code, you'll notice:
>
> - both use SQLite with a local file db, which is ok
> - Bottle uses SQLite driver directly
> - Pyramid uses SQLAlchemy which incurs significant overhead
>
> You can use database drivers directly in Pyramid. You don't need sqlalchemy,
> or transaction extensions, they are not required by Pyramid, just a chosen
> default.
ways to use SQLAlchemy to minimze the overhead. That comes at the
expense of convenience, of course, so you'd want to do a side-by-side
comparison of a typical task to see how much the overhead matters.
The biggest performance improvement is when you do bulk queries in the
database. One SQL statement that does some calculations and returns a
small number of result records. That avoids the overhead of loading
every record into some Python data type. You can also do bulk updates
and deletes. Optimized bulk processing at the C level is a feature of
all SQL-compatible databases. It often is not a feature of non-SQL
databases. For instance, CouchDB can perform a Javascript query and
return some records, but I don't know if it's any faster than going
through all the records in Python.
The second-largest optimization is to use SQLAlchemy's SQL builder
level rather than the ORM level. You can send it a SQL string to
execute, or use the builder methods to construct the SQL statement.
This essentially runs at the same speed as raw DB-API because it's
doing the same thing. The result is an iterable of RowProxy's, which
incur some minimal overhead. It you access fields by key rather than
attribute (x[0] or x["foo"] vs x.foo), it's supposed to be faster.
RowProxy uses lazy evaluation, so it avoids processing the underlying
row tuple except as necessary.
The ORM has to instantiate a Python object for every record, and keep
track of which objects have had their attributes changed in Python.
But again it does lazy evaluation, so it's not like it sets every
attribute on initialization. Recent versions of SQLAlchemy also have
a feature that you can construct a query using the ORM methods, but if
it's a query on certain fields rather than on an entire ORM object, it
returns RowProxy's just like the SQL level, so it bypasses most of the
ORM's overhead.
But again, you should do a side-by-side comparision to see how much
the overhead actually is, because sometimes it surprises you. I have
an import routine that reads 10,000+ records from CSV files and puts
them in an empty database, and it takes 30 seconds to run either with
or without the ORM. On the other hand, I have some log-processing
scripts that process hundreds of thousands or millions of records, and
the speedup is significant if I switch from ORM-level processing to
SQL-level processing. (But again, I can use the ORM methods to
construct the queries; I just avoid returning ORM objects or inserting
ORM objects.)
The third thing you can do is to hold a long-lived connection
throughout the application, rather than letting the engine check out a
connection on every query. That avoids the overhead of the connection
pool. But that probably makes little difference. The purpose of the
connection pool is to avoid the larger overhead of actually connecting
to the database on every query. That's slow on some databases like
PostgreSQL, but fast on others like SQLite. So the pool actually
improves performance, and raw DB-API does not have a connection pool.
This again depends on your application. A short-lived, single-threaded
utility can just hold a connection for simplicity. But a multithreaded
web application really benefits from a pool, so that you don't have to
manage connections. (Or at most, you hold a connection open for a
single function or single request.)
SQLAlchemy is wonderful because it's multi-level: you can give it SQL
strings, use the SQL builder, or use the ORM, depending on the
application or even in different places in the same application.
Python never had a multi-level database library before SQLAlchemy. I
don't know how common it is in other programming languages. Michael
Bayer also writes excellent documentation. (He wrote SQLAlchemy and
Mako.) So I would definitely recommend using it.
The other issue is, raw benchmarks of X framework does N requests per
hour are usually unrealistic. They use an empty application to measure
the framework's overhead. But in the real world of complex
applications using databases and performing calculations, the overhead
of the framework is dwarfed by the overhead of the application code.
If you have a very small, simple application like Twitter with a huge
number of request, then the framework's performance would be close to
the benchmark. Otherwise it will degrade in ways that aren't framework
specific. (I.e., they'd be the same in Pyramid or Flask.) If your
application is so high-volume that it approaches the hardware's
capacity, then you should look at parallel servers as well as
different frameworks or languages. It may be that the cost of a second
server is less than the programming-time cost or inconvenience of
using a "simple, streamlined" framework or C-like language. Unless the
application is very simple, in which case a minimalistic framework may
be perfect for it.
In terms of Python WSGI applications, there are two separate
overheads: the WSGI server, and the framework/application. The
CherryPy server is considered the most robust at high loads, compared
to other multhithreaded Python servers. You can use it with Pyramid,
just set the "[server:main]" section in the INI file. Asynchronous
servers may have higher performance than mulththreaded, but the
difficulty of making an application asynchronous-safe may outweigh the
advantages. You can also use a module like mod_wsgi to avoid the
overhead of a separate WSGI HTTP server.
--
Mike Orr <slugg...@gmail.com>
Mike Orr
> Optimized bulk processing at the C level is a feature of
> all SQL-compatible databases.
--
Mike Orr <slugg...@gmail.com>
Jonathan Vanasco
1) Just to add on Overhead stuff as mentioned above, I've honestly
never seen a webapp that connects to a Database ( Mysql, Postgresql,
Couch/Mongo/etc ) or exists in a SOA talking to other daemons/apis,
where the webapp was the bottleneck. In my experience, it's always
been the database or internal APIs holding stuff up, or the frontend
server not being able to handle concurrent connections well ( and not
getting them to the backend fast enough ). I'm pretty much not
worried by the overhead of things like SqlAlchemy -- there are better
places to optimize... like Pyramid itself, which is incredibly
unlikely to be a bottleneck itself.
> I have been primarily a C/C++ system level programmer and architected and developed large Telecom systems (carrier grade) requiring great performance levels. Currently looking into Python and corresponding web frameworks for our platform.
looking for a website, i'd be focused on Erlang.
> How is the performance of Paster Web server as compared to something like lighttpd (which is supposedly a light weight server)? If lighttpd, then I would need to embed Pylons/Pyramid there. (I have embedded Pylons with Apache but haven’t checked the performance).
never really being all that great. Nginx is the current leader in
light http servers.
You wouldn't run python within the http server, but use nginx to
either proxy to a server or whatever handles wsgi.
> Regarding DB, SQLAlchemy can be a choice for SQL based DB servers. Considering something like MongoDB, not sure how would SQLAlchemy help there. But certainly if I have my own DB access library using native APIs, probably a better performance can be achieved.
tradeoff -- you give up a marginally faster runtime ( vs a native API,
a custom framework, programming in C ) for significantly faster times
in your release schedule, management cycle, and a much lower cost-to-
change and iterate.
So yes, you can go crazy trying to optimize the theoretical max
performance of an application - or you can choose something like
Pyramid and SqlAlchemy and optimize the actual performance of your
development team. I prefer the latter.
Vlad K.
> SQLAlchemy's overhead is not necessarily significant, and there are
> ways to use SQLAlchemy to minimze the overhead. That comes at the
> expense of convenience, of course, so you'd want to do a side-by-side
> comparison of a typical task to see how much the overhead matters.
SQLAlchemy, not SQLA as a whole. I use it exclusively, perfect for rapid
prototyping and in majority of cases more than suitable, and intend to
push certain database logic down to the DB level for performance.
> In terms of Python WSGI applications, there are two separate
> overheads: the WSGI server, and the framework/application. The
> CherryPy server is considered the most robust at high loads, compared
> to other multhithreaded Python servers. You can use it with Pyramid,
> just set the "[server:main]" section in the INI file. Asynchronous
> servers may have higher performance than mulththreaded, but the
> difficulty of making an application asynchronous-safe may outweigh the
> advantages. You can also use a module like mod_wsgi to avoid the
> overhead of a separate WSGI HTTP server.
in my case have clearly seen its superiority to Apache/mod_wsgi. I have
no experience with CherryPy.
http://nichol.as/benchmark-of-python-web-servers
--
.oO V Oo.
Biswas, Pinakee
Thanks for the detailed information. It certainly helps.
We would certainly consider the information on SQLAlchemy for DB related performance (there are certainly other mechanisms to improve performance like bulk queries, probably stored procedures etc.).
The platform we are building is not simple - it’s a media (mostly video) based content management and delivery platform with multiple features.
As far as my understanding goes on Pylons/Pyramid, the requests handling and delivery (to proper controller and action), management of threads (not sure if this is applicable) is handled by the framework . The HTTP protocol level transaction handling, detailed implementation of the stack etc. are handled by the web server/framework.
The job remaining then by the application (or the application developer) is processing the requests and building the response (which also the framework helps in building with various libraries).
As you mentioned, the application development would certainly play a role in the performance of the overall system.
I agree that the ease of development is important but probably not at the cost of system performance.
While choosing the overall framework for our platform, we just wanted to make sure that we are choosing the right framework.
Thanks,
Pinakee
Please don't print this e-mail unless you really need to, this will preserve trees on planet earth.
The information contained in this message (including any attachments) is confidential and may be privileged. If you have received it by mistake please notify the sender by return e-mail and permanently delete this message and any attachments from your system. Please note that e-mails are susceptible to change and malwares. VVIDIA COMMUNICATIONS PVT LTD. (including its group companies) shall not be liable for the improper or incomplete transmission of the information contained in this communication nor for any delay in its receipt or damage to your system.
-------------------------------------------------------------------------------------------------------------------------Disclaimer----------------------------------------------------------------------------------------------------------
From: pylons-...@googlegroups.com [mailto:pylons-...@googlegroups.com] On Behalf Of Mike Orr
Sent: 07 June 2012 01:41
To: pylons-...@googlegroups.com
Subject: Re: Pylons/Pyramid Performance
Biswas, Pinakee
Thanks for the information.
I thought Youtube used lighttpd.
Thanks,
Pinakee
Please don't print this e-mail unless you really need to, this will preserve trees on planet earth.
The information contained in this message (including any attachments) is confidential and may be privileged. If you have received it by mistake please notify the sender by return e-mail and permanently delete this message and any attachments from your system. Please note that e-mails are susceptible to change and malwares. VVIDIA COMMUNICATIONS PVT LTD. (including its group companies) shall not be liable for the improper or incomplete transmission of the information contained in this communication nor for any delay in its receipt or damage to your system.
-------------------------------------------------------------------------------------------------------------------------Disclaimer----------------------------------------------------------------------------------------------------------
Mengu
in when it comes to performance. 1) TurboGears. 2) Pyramid.
last year, i have written a statistics application (a chartbeat like
app) in Pyramid. i have used only 1 machine that had 4 (or 8, don't
remember the exact count) cores and 16 gb ram. with apache & mod_wsgi
i could handle ~40K reqs/sec. with nginx & uwsgi i could handle
50K-55K reqs/sec. since that day i always deploy my applications with
nginx & uwsgi. :)
let's talk about a portal. a portal is read-heavy application. you can
have a simple replication, one master one slave. you can put redis in
front of the database so the reads are faster or you can just use
varnish to cache the pages. other than that it's your server's or
servers' capabilities.
On Jun 6, 7:10 am, "Biswas, Pinakee" <pina...@vvidiacom.com> wrote:
> Hi,
>
> As mentioned earlier, we are planning to use Pylons/Pyramid for our
> platform.
>
> Could you please let me know if there is any performance statistics
> available for Pylons/Pyramid?
>
> Is there any major portal with high traffic that is based on Pylons/Pyramid?
> I know one major Portal (with almost 12000 views per sec and few petabytes
> of monthly bandwidth usage) uses Python but not sure if it uses any web
> framework.
>
> Thanks,
>
> Pinakee Biswas
>
> Director & CTO
>
>
> Just watch it !
>
> yGSM Mobile (Voice & Data): +91-982-016-7979
>
> (Tel: +91-22-26733737 I (Fax:+91-22-26732828
>
> <http://www.vvidiacom.com/>http://www.vvidiacom.com
> P Please don't print this e-mail unless you really need to, this will
> preserve trees on planet earth.
>
> ----------------------------Disclaimer------------------------------------- -
> --------------------------------------------------------------------------- -
> --------------------------------------------------------------------------- -
> ---------
>
> The information contained in this message (including any attachments) is
> confidential and may be privileged. If you have received it by mistake
> please notify the sender by return e-mail and permanently delete this
> message and any attachments from your system. Please note that e-mails are
> susceptible to change and malwares. VVIDIA COMMUNICATIONS PVT LTD.
> (including its group companies) shall not be liable for the improper or
> incomplete transmission of the information contained in this communication
> nor for any delay in its receipt or damage to your system.
>
> --------------------------------------------------------------------------- -
> ---------------------------------------------Disclaimer-------------------- -
> --------------------------------------------------------------------------- -
> ---------
>
> 2KViewDownload
>
> image002.png
> < 1KViewDownload
Mike Orr
> The platform we are building is not simple - it’s a media (mostly video) based content management and delivery platform with multiple features.
if you're editing/processing videos, the multimedia aspects of those.
Neither of those has much to do with the choice of framework. Of
course, you won't want to deliver large files through the framework
where it has to read the entire file into Python, but rather use
X-SendFile or a media server or such for them.
> As far as my understanding goes on Pylons/Pyramid, the requests handling and delivery (to proper controller and action), management of threads (not sure if this is applicable) is handled by the framework . The HTTP protocol level transaction handling, detailed implementation of the stack etc. are handled by the web server/framework.
> The job remaining then by the application (or the application developer) is processing the requests and building the response (which also the framework helps in building with various libraries).
It loads an application instance into each worker thread. Pyramid
comes along as part of the application. Pyramid does nothing with
threads; it runs inside a thread. The only thing it has to do is be
thread-safe.
HTTP issues are divided between the WSGI server and application. The
WSGI server converts the HTTP headers to a basic Python data structure
and may modify them. The application (Pyramid, using WebOb) converts
them to another Python structure and may modify them further.
The application's responsibility, as you say, is to interpret a
request and decide a response.
--
Mike Orr <slugg...@gmail.com>
Jonathan Vanasco
2005-2008. It was the first high performance lightweight server ,
with documentation in English. It also had a lot of memory leaks and
other bugs. But it had high performance and you could just have a
cron-job kill&respawn it hourly.
Nginx, at that time, was just as robust and had no major issues --
except the docs were entirely in Russian and the mailing list /
community was mostly in broken English.
Over time, the Nginx community came up with English docs & a great
wiki. Many of the Lighttpd users were frustrated and dropped lighttpd
for nginx or Apache2. Quoting nginx.com ( http://www.nginx.com/company.html
) """ It is used by 10.15% of all web sites and over 25% of the top
1,000 busiest web sites on the Internet, including Facebook, Zappos,
Groupon, LivingSocial, Hulu, TechCrunch, Dropbox and WordPress."""
lighttpd isn't even on those charts.
No one publicly knows what YouTube is using for videos today. Their
main site is in Apache, and their videos have a cryptic "gvs 1.0"
header, which could stand for "google video server" or the name of the
CDN application. Whatever the cause, I strongly doubt that Lighttpd is
still in use. The site has been barely updated over the past 5 years
( the benchmarks are even 5years old)
Also worth nothing that the front page of lighttpd.net claims
YouTube , WikiPedia and Meebo -- however meebo shows nginx, WikiPedia
shows apache on html and "Sun-Java-System-Web-Server/7.0" on assets.
Biswas, Pinakee
Thanks for the information. It really helps.
We would certainly look into nginx.
Could you please suggest if embedding Python applications (including Pylons/Pyramid) in the Web Server (like nginx) would be the best approach (if it is possible to do so with nginx)? Or redirecting the traffic from the web server (like nginx) to the WSGI server (With Pylons/Pyramid application) would be a good approach? We would be looking at this from the performance point of view.
I think Interprocess communication sometimes creates an overhead unless we have a distributed system spawning across multiple servers - wherein we can have load distribution across multiple instances. Something like the web server (nginx) distributing the load across multiple instances of the WSGI servers (if it is possible to do so) - without obviously creating a bottleneck at the web server and the web server is just a simple process routing traffic between the WSGI servers/application and the network.
Looking forward to your thoughts...
Thanks,
Pinakee
FE- Mail: pin...@vvidiacom.com I IWeb: http://www.vvidiacom.com
Please don't print this e-mail unless you really need to, this will preserve trees on planet earth.
----------------------------Disclaimer-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The information contained in this message (including any attachments) is confidential and may be privileged. If you have received it by mistake please notify the sender by return e-mail and permanently delete this message and any attachments from your system. Please note that e-mails are susceptible to change and malwares. VVIDIA COMMUNICATIONS PVT LTD. (including its group companies) shall not be liable for the improper or incomplete transmission of the information contained in this communication nor for any delay in its receipt or damage to your system.
-------------------------------------------------------------------------------------------------------------------------Disclaimer----------------------------------------------------------------------------------------------------------
-----Original Message-----
Sent: 08 June 2012 01:57
To: pylons-discuss
Subject: Re: Pylons/Pyramid Performance
Biswas, Pinakee
Hi Mike,
Thanks for the detailed information.
With so many options available, it becomes a bit confusing to conclude on a specific design J:
1. Whether to embed the Pylons/Pyramid application in a light weight web server (like nginx as suggested by Jonathan) if possible
2. Or to redirect the network HTTP traffic from the web server to the WSGI servers (pastehttpserver, cherrypy etc.)?
Which one would be the best approach in terms of performance? As far as my knowledge goes, I think 2 would be good if we have distributed systems wherein the webserver distributes the load across multiple instances of the WSGI server (given that the webserver is a light weight process just routing traffic between the WSGI servers and network).
Would really appreciate your suggestion regarding the above…
Please don't print this e-mail unless you really need to, this will preserve trees on planet earth.
----------------------------Disclaimer-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The information contained in this message (including any attachments) is confidential and may be privileged. If you have received it by mistake please notify the sender by return e-mail and permanently delete this message and any attachments from your system. Please note that e-mails are susceptible to change and malwares. VVIDIA COMMUNICATIONS PVT LTD. (including its group companies) shall not be liable for the improper or incomplete transmission of the information contained in this communication nor for any delay in its receipt or damage to your system.
-------------------------------------------------------------------------------------------------------------------------Disclaimer----------------------------------------------------------------------------------------------------------
-----Original Message-----
From: pylons-...@googlegroups.com [mailto:pylons-...@googlegroups.com] On Behalf Of Mike Orr
Sent: 07 June 2012 23:07
To: pylons-...@googlegroups.com
Subject: Re: Pylons/Pyramid Performance
On Wed, Jun 6, 2012 at 9:48 PM, Biswas, Pinakee <pin...@vvidiacom.com> wrote:
--
Mike Orr
> With so many options available, it becomes a bit confusing to conclude on a
> specific design J:
>
> 1. Whether to embed the Pylons/Pyramid application in a light weight
> web server (like nginx as suggested by Jonathan) if possible
>
> 2. Or to redirect the network HTTP traffic from the web server to the
> WSGI servers (pastehttpserver, cherrypy etc.)?
>
>
>
> Which one would be the best approach in terms of performance? As far as my
> knowledge goes, I think 2 would be good if we have distributed systems
> wherein the webserver distributes the load across multiple instances of the
> WSGI server (given that the webserver is a light weight process just routing
> traffic between the WSGI servers and network).
is called a reverse proxy, and from what I've read there's significant
overhead in converting one HTTP request to a different HTTP request
for forwarding. FastCGI/scgi/mod_wsgi, etc, are supposed to be more
efficient protocols than proxying.
I use proxying because it's simpler and more foolproof. I can use a
text web browser on the server to contact the WSGI port directly for
testing/troubleshooting, to tell whether a blockage is in the Python
code or in Apache. I can run functional tests (Twill) on it just like
the development environment. My sites are moderate usage so I don't
have to worry about hardware capacity.
For load distribution, you'd use a real load balancer at the front
end. Each backend server would run its own instance of Nginx/Apache.
So it doesn't change anything re WSGI choices on those servers.
--
Mike Orr <slugg...@gmail.com>