[Lift] dealing with accumulated LiftSessions

[harryh]

I'm starting to run into problems where my app servers are having
memory issues at least partially due to the fact that many thousands
of LiftSessions are being created and never actually used for more
than a single request. The primary cause of this is, of course, bots
and web crawlers. They hit a single page, a session gets created, and
then never used again.

What might we be able to do about this? A couple of thoughts:

1) requests from known IPs, or with known User-Agents throw away the
LiftSession after the single request

2) a max_sessions setting that puts LiftSessions in a LRU cache, and
throws away old ones when max_sessions is exceeded

3) LiftSessions that have only been used once time out much faster
than other LiftSessions.

-harryh

--
You received this message because you are subscribed to the Google Groups "Lift" group.
To post to this group, send email to lif...@googlegroups.com.
To unsubscribe from this group, send email to liftweb+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.

[David Pollak]

On Wed, Apr 28, 2010 at 6:00 PM, harryh <har...@gmail.com> wrote:

I'm starting to run into problems where my app servers are having
memory issues at least partially due to the fact that many thousands
of LiftSessions are being created and never actually used for more
than a single request.  The primary cause of this is, of course, bots
and web crawlers.  They hit a single page, a session gets created, and
then never used again.

What might we be able to do about this?  A couple of thoughts:

I think there could be a hybrid of these ideas.  Basically record the IP and user agent associated with the creation of the session.

Then we'll add a function that could be run on the list of sessions every 10 seconds or so that will generate a list of sessions to be purged.

What do you think?

Want to open a ticket and I'll crank some stuff out tomorrow morning.

Thanks,

David (who has been thinking about this very issued for 6 months or so)
 

1) requests from known IPs, or with known User-Agents throw away the
LiftSession after the single request

2) a max_sessions setting that puts LiftSessions in a LRU cache, and
throws away old ones when max_sessions is exceeded

3) LiftSessions that have only been used once time out much faster
than other LiftSessions.

-harryh

--
You received this message because you are subscribed to the Google Groups "Lift" group.
To post to this group, send email to lif...@googlegroups.com.
To unsubscribe from this group, send email to liftweb+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics

[harryh]

> I think there could be a hybrid of these ideas.  Basically record the IP and
> user agent associated with the creation of the session.
>
> Then we'll add a function that could be run on the list of sessions every 10
> seconds or so that will generate a list of sessions to be purged.
>
> What do you think?

I think that sounds great. I also think that recording the # of times
the LiftSession has been used would be useful. LiftSessions only used
to load a single page should expire very fast I think.

> Want to open a ticket and I'll crank some stuff out tomorrow morning.

https://www.assembla.com/spaces/liftweb/tickets/499-cleaning-up-single-use-liftsessions

> David (who has been thinking about this very issued for 6 months or so)

Indeed, I'm pretty sure we've even talked about this before.