Load capabilities with default pool_size and max_overflow
Fotinakis
What is (or what do you think is) the load that SQLAlchemy can handle
with the default engine options of pool_size=5 and max_overflow=10?
The application I'm working on has the potential for bursts of
thousands of requests in a few seconds, so I am concerned about load.
Is it appropriate to increase these values to some arbitrary number? I
see that you can set max_overflow to -1 so that it creates as many
connections as needed, but do you think that would be necessary (or
bad practice)? Thanks for any advice about this, I'm trying to
understand the nature of connections and the QueuePool implementation
to avoid load problems when we go live. Any simple way to script
exceeding the QueuePool limit quickly for load testing?
Thanks for the help, much appreciated.
And also, I've wanted to throw out a huge thank you to the
developers ... on top of SQLAlchemy being extremely robust and
powerful, the SQLAlchemy documentation is impeccable. Better than most
all other open source software ... *awesome* ... so, thank you.
Andreas Jung
--On 15. Juli 2008 22:32:35 -0700 Fotinakis <fotin...@gmail.com> wrote:
>
> Hello SQLAlchemists,
>
> What is (or what do you think is) the load that SQLAlchemy can handle
> with the default engine options of pool_size=5 and max_overflow=10?
> The application I'm working on has the potential for bursts of
> thousands of requests in a few seconds, so I am concerned about load.
Default settings are called that way because they make sense to possibly
80% of all use-cases. Don't expect that a high-traffic environment works
without tuning. So the answer is basically: unlikely.
>
> Is it appropriate to increase these values to some arbitrary number?
Those parameters are exposed as part of the API for performing such a
tuning?!
-aj
Michael Bayer
On Jul 16, 2008, at 1:32 AM, Fotinakis wrote:
>
> Hello SQLAlchemists,
>
> What is (or what do you think is) the load that SQLAlchemy can handle
> with the default engine options of pool_size=5 and max_overflow=10?
> The application I'm working on has the potential for bursts of
> thousands of requests in a few seconds, so I am concerned about load.
the biggest issue on your mind then should be how many concurrent
requests your system would be expected to handle. The connection pool
generally needs to be sized according to this.
> Is it appropriate to increase these values to some arbitrary number? I
> see that you can set max_overflow to -1 so that it creates as many
> connections as needed, but do you think that would be necessary (or
> bad practice)?
-1 is probably not a great idea since the database itself has a max
number of connections configured, not to mention your app server has
only a finite amount of memory in which to handle connections. you'd
want to do some memory analysis and see how much memory a single
connection takes up.