Problem with default Connection pool/CP for use with lift?
Jeppe Nejsum Madsen
We ran into an issue today. Basically, our postgres volume went into
readonly mode, making database access...hmmm lets say less than perfect
:-)
We quickly revived the db server, but this surfaced a problem which I
think is in the default connection pool of the generated Lift archetypes
(which I haven't got around to changing...yet!):
The connections created to the db are always put back into the pool and
never tested for validity. So if the db server crashes and connections
are already in the pool, it seems like they are reused but they'll fail
when a new db server instance is started. Only recourse seems to be to
restart the container (Jetty in our case)
Most J2EE servers has some built-in connection pool functionality that
will test connections etc and create new if needed.
What are people using for their Lift apps in the simpler container-only
setup (e.g. Jetty?)
/Jeppe
David Pollak
case x :: xs => try {
x.setAutoCommit(false)
Full(x)
} catch {
case e => try {
pool = xs
poolSize = poolSize - 1
x.close
newConnection(name)
} catch {
case e => newConnection(name)
}
}
If the setAutoCommit(false) line fails, the connection is returned to the pool. You can put other logic here.
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics
Jeppe Nejsum Madsen
> At least the most recent version of Boot.scala has reasonable testing of
> connection validity:
>
> case x :: xs => try {
> x.setAutoCommit(false)
> Full(x)
> } catch {
> case e => try {
> pool = xs
> poolSize = poolSize - 1
> x.close
> newConnection(name)
> } catch {
> case e => newConnection(name)
> }
> }
>
> If the setAutoCommit(false) line fails, the connection is returned to the
> pool. You can put other logic here.
Yes, I can always put a "SELECT 1" in there, but was curious what others
are doing in this regard (if anything)....
/Jeppe
Jeppe Nejsum Madsen
> At least the most recent version of Boot.scala has reasonable testing of
> connection validity:
>
> case x :: xs => try {
> x.setAutoCommit(false)
> Full(x)
> } catch {
> case e => try {
> pool = xs
> poolSize = poolSize - 1
> x.close
> newConnection(name)
> } catch {
> case e => newConnection(name)
> }
> }
>
> If the setAutoCommit(false) line fails, the connection is returned to the
> pool. You can put other logic here.
Ok, I just made this change and added some logging around new/Release. I
was a little surprised to see that newConnection is called three times
for each request. Is this intentional?
I have S.addAround(DB.buildLoanWrapper) in boot, which I assumed would
wrap each request with a transaction and thus use the same connection
for the duration of the request? I've verified that the same connection
is used on two requests executing at the same time which sounds a bit
scary....
Did I miss something?
/Jeppe
Jeppe Nejsum Madsen
> David Pollak <feeder.of...@gmail.com> writes:
>
>> At least the most recent version of Boot.scala has reasonable testing of
>> connection validity:
>>
>> case x :: xs => try {
>> x.setAutoCommit(false)
>> Full(x)
>> } catch {
>> case e => try {
>> pool = xs
>> poolSize = poolSize - 1
>> x.close
>> newConnection(name)
>> } catch {
>> case e => newConnection(name)
>> }
>> }
>>
>> If the setAutoCommit(false) line fails, the connection is returned to the
>> pool. You can put other logic here.
>
> Ok, I just made this change and added some logging around new/Release. I
> was a little surprised to see that newConnection is called three times
> for each request. Is this intentional?
Sorry, to keep adding to this thread, but I looked a little more in
the archetype implementation of ConnectionManager, and to me, it seems
like it is broken in several ways and will eventually lead to an OOME:
I think the logic is that pool should contain the list of connections
available for use by a request and not all the connections created?
But it seems like the implementation mix those two views:
First time through, pool is Nil so we match the first case statement,
create a new connection _AND ADD IT TO POOL_. First error: we return
the connection (ie it is in use) but still add it to the list of free
connections. Another request coming in while we process this request,
will get the same connection.
When releasing the connection, we add the released connection to the pool. Fine.
Next time, pool is not Nil so we match the x::xs case and return x.
Second error: we don't update pool to xs, x is still available for any
request coming in while we're using this connection.
And since connections are never removed from pool under normal
circumstances but always added on release, the pool list just keeps
growing.....
I'll post an updated version soon....
/Jeppe