Explicit Session un/locking?

[Austin Steed]

Hey All!

I'm having trouble understanding how explicit session locking & unlocking works.

Currently I have a problem where with implicit session locking, a lot of the pages of my webapp kind of freeze because w/ implicit cherrypy only processes the next request once the session is unlocked.

That said, I'm getting a ton of errors when I try to use explicit:

cherrypy.session.acquire_lock()

cherrypy.session.release_lock()

When do I use these? When reading? Only when modifying the cherrypy.session data? Only when saving? When loading?

With explicit locks do I need to load/save session data or is that still done implicitly?

I'm getting this error:

ERROR:cherrypy.error.49758160:[23/May/2014:10:28:47] HTTP Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 680, in respond

    self.hooks.run('before_finalize')

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 114, in run

    raise exc

AssertionError: The session was saved without being locked.  Check your tools' priority levels.

most of the time, after fiddling around with it I was getting an error about loading.

I've tried changing my auth tools priority level to be higher and lower than the default 50. Still no luck.

Any help would be appreciated! I'm basically just trying to figure out how to use explicit locks. Or maybe if implicit with explicitly calling release_lock() might be better.

I have about 200 exposed methods, about 30% of them are long running ( between 1-30 seconds, typically 1 second) b/c they are querying a large database.

Thank you!

-Austin

[Austin Steed]

Any help? Is there more information I can post to be of more use?

-Austin

[Walter Woods]

Hi!  I believe you can use:

cherrypy.session.save()

To both release the lock AND save your data.  What I would recommend (as it is easiest) is that you continue to use implicit locking in general, but pull what you need to out of the session and then call this method before any "long term" calculations in your handlers.  Then other handlers may execute on the same session, which is now unlocked.

As for the error you're seeing, I seem to recall having seen that issue in a handler for a cherrypy version that was kind of old, but I expect it would be solved by now.  Which version are you using?

Hope that helps,
Walt

--
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cherrypy-user...@googlegroups.com.
To post to this group, send email to cherryp...@googlegroups.com.
Visit this group at http://groups.google.com/group/cherrypy-users.
For more options, visit https://groups.google.com/d/optout.

[Austin Steed]

I'm using the latest version in the pip repository, cherrypy '3.3.0', for python 2.7.6. Should I use implicit even if almost every single handler does not use the session data? (I do have a login/auth tool I call that uses it for all handlers)

If I use implicit, I would need to go through all of my handlers and call  cherrypy.session.save() or cherrypy.session.release_lock() right at the beginning of the handler correct?

I only have like 5 handlers out of the 200 that actually use the session data, which is why I was thinking explicit might be a better option?

Thank you for your reply!

-Austin

[Walter Woods]

Let me clarify - if you're sure that only 5 / 200 handlers use session data, it does sound like it would be better to use explicit session locking.  However, in my experience, I have had more difficulty getting explicit locking to work with the default tools for whatever reason.

Also - are you sure you only need session data in only 5 handlers?  Do you have users that log in?  Bear in mind that recognizing a logged in user also uses the session.

Happy to help,
Walt

[Austin Steed]

I do use sessions, and you're right all the handlers do need to be authenticated prior to executing.

My authentication is done via a decorator:

# authentication decorator

def auth_check():

#     cherrypy.session.acquire_lock()

    authenticated = cherrypy.session.get('authenticated', False)

#     cherrypy.session.save()

    

    if not authenticated:

        raise cherrypy.HTTPRedirect("/account/login")

cherrypy.tools.auth_check = cherrypy.Tool("before_handler", auth_check, priority=60)

Then above each exposed handler that needs to be authenticated do this:

    @cherrypy.tools.auth_check()

    def index(self):

I've tried making the sessions explicit and un-commenting the code above, however I still get the save error mentioned in my first post.

Is this correct or am I doing something wrong here?

Thanks again!

-Austin

[Walter Woods]

Then using implicit locking.  Are you saying it still breaks even if "save" is used to unlock?  Let me look at the source... it looks fine to me.  Looks like you're using File sessions, right?

If you use implicit locking, and only call save() AT MOST ONCE per handler, then you should be fine.

[Austin Steed]

Correct, I'm using file sessions.

With implicit locking on and calling "cherrypy.session.save()" only once (in the auth_check() tool showed above).

I get the following:

[29/May/2014:14:15:39]  Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 104, in run

    hook()

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 63, in __call__

    return self.callback(**self.kwargs)

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 784, in save

    cherrypy.session.save()

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 253, in save

    self._save(expiration_time)

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 515, in _save

    assert self.locked, ("The session was saved without being locked.  "

AssertionError: The session was saved without being locked.  Check your tools' priority levels.

ERROR:cherrypy.error.35325776:[29/May/2014:14:15:39]  Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 104, in run

    hook()

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 63, in __call__

    return self.callback(**self.kwargs)

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 784, in save

    cherrypy.session.save()

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 253, in save

    self._save(expiration_time)

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 515, in _save

    assert self.locked, ("The session was saved without being locked.  "

AssertionError: The session was saved without being locked.  Check your tools' priority levels.

[29/May/2014:14:15:39] HTTP Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 674, in respond

    self.hooks.run('before_finalize')

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 114, in run

    raise exc

AssertionError: The session was saved without being locked.  Check your tools' priority levels.

ERROR:cherrypy.error.35325776:[29/May/2014:14:15:39] HTTP Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 674, in respond

    self.hooks.run('before_finalize')

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 114, in run

    raise exc

AssertionError: The session was saved without being locked.  Check your tools' priority levels.

I'm not calling any other cherrpy.session.* methods, nor is it ever modified after the auth_check() is executed.

I appreciate your help!

-Austin

[Walter Woods]

https://bitbucket.org/cherrypy/cherrypy/src/60e2a6047e05be150c102b633715730b87593a08/cherrypy/_cptools.py?at=default#cl-309

Line 309 - session locking also has priority 60.  Change your tool priority so it happens after that (I assume that means a lower priority like 59, it's been awhile since I messed with that).

[Austin Steed]

Ok, I've tried it with priority=10, 40, 50, 60, 80, and 99. 

The sessions below 60 priority:

[29/May/2014:14:43:09]  Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 104, in run

    hook()

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 63, in __call__

    return self.callback(**self.kwargs)

  File "/home/trdis/trdis/tradas/web_ui/pages.py", line 47, in auth_check

    authenticated = cherrypy.session.get('authenticated', False)

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 348, in get

    self.load()

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 268, in load

    data = self._load()

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 497, in _load

    assert self.locked, ("The session load without being locked.  "

AssertionError: The session load without being locked.  Check your tools' priority levels.

ERROR:cherrypy.error.43662800:[29/May/2014:14:43:09]  Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 104, in run

    hook()

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 63, in __call__

    return self.callback(**self.kwargs)

  File "/home/trdis/trdis/tradas/web_ui/pages.py", line 47, in auth_check

    authenticated = cherrypy.session.get('authenticated', False)

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 348, in get

    self.load()

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 268, in load

    data = self._load()

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 497, in _load

    assert self.locked, ("The session load without being locked.  "

AssertionError: The session load without being locked.  Check your tools' priority levels.

[29/May/2014:14:43:09] HTTP Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 667, in respond

    self.hooks.run('before_handler')

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 114, in run

    raise exc

AssertionError: The session load without being locked.  Check your tools' priority levels.

ERROR:cherrypy.error.43662800:[29/May/2014:14:43:09] HTTP Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 667, in respond

    self.hooks.run('before_handler')

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 114, in run

    raise exc

AssertionError: The session load without being locked.  Check your tools' priority levels.

The sessions at or above 60 priority yield: 

[29/May/2014:14:41:39]  Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 104, in run

    hook()

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 63, in __call__

    return self.callback(**self.kwargs)

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 784, in save

    cherrypy.session.save()

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 253, in save

    self._save(expiration_time)

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 515, in _save

    assert self.locked, ("The session was saved without being locked.  "

AssertionError: The session was saved without being locked.  Check your tools' priority levels.ERROR:cherrypy.error.67312720:[29/May/2014:14:41:39]  Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 104, in run

    hook()

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 63, in __call__

    return self.callback(**self.kwargs)

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 784, in save

    cherrypy.session.save()

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 253, in save

    self._save(expiration_time)

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/sessions.py", line 515, in _save

    assert self.locked, ("The session was saved without being locked.  "

AssertionError: The session was saved without being locked.  Check your tools' priority levels.[29/May/2014:14:41:39] HTTP Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 674, in respond

    self.hooks.run('before_finalize')

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 114, in run

    raise exc

AssertionError: The session was saved without being locked.  Check your tools' priority levels.ERROR:cherrypy.error.67312720:[29/May/2014:14:41:39] HTTP Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 674, in respond

    self.hooks.run('before_finalize')

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 114, in run

    raise exc

AssertionError: The session was saved without being locked.  Check your tools' priority levels.

I think priority executes based on ascending order, executing smallest first, right?

Thanks!

-Austin

[Walter Woods]

Looks that way, good investigation :)   Looking a few lines above where I cited (d'oh), you want "early" locking, not "implicit".  Implicit apparently executes after the other tools.  Sorry about that.

[Austin Steed]

Is "early" an option I put in the config? I don't see it in the documentation, only "implicit" and "explicit"

-Austin

--
You received this message because you are subscribed to a topic in the Google Groups "cherrypy-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cherrypy-users/bSETdqxB--U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cherrypy-user...@googlegroups.com.

[Walter Woods]

Docs aren't complete there apparently, yes use "early" in place of "implicit"

[Austin Steed]

Thank you! Is there any place I can find more information on "early"? When exactly does it lock/unlock? How much earlier than implicit?

I tried to find the "early", "implicit", "explicit" arguments in the cherrypy source but I'm not really sure where to look.

Thanks again!

-Austin

[Walter Woods]

Look at the source code I cited a few replies back :) that's the exact spot

[Austin Steed]

Ahh thanks! I can't believe I glossed over that!

Am I understanding this correctly?

        hooks.attach('before_finalize', _sessions.save)
        hooks.attach('on_end_request', _sessions.close)

So at the return of every handler, 'before_finalize' the session is saved?

Then after finalizing but before the request ends, the session is closed?

So if I were to do explicit locking I would need to call acquire_lock() at the end of every handler method? So that the lock was acquired and it could be saved?

Seems like a lot of work being done even when the session is never being modified?

Thanks!

-Austin

[Walter Woods]

> So at the return of every handler, 'before_finalize' the session is saved?

> Then after finalizing but before the request ends, the session is closed?

Right.

> So if I were to do explicit locking I would need to call acquire_lock() at the end of every handler method?

Not right :)  Those two functions both only do something if the session was locked / loaded respectively.  Performance wise you lose a few function calls, but compared to your client code that is almost certainly negligible.  It doesn't need to actually load the session if it wasn't already.

[Austin Steed]

Ahh ok, I see in the sessions.py source file that it only loads the sessions on get/set/del/pop/etc, i.e. the session only loaded if I directly access the cherrypy.session object.

Okay well I'll play around with it a bit more. Though the "early" mode seems to work "good enough" for now, though there are a few hiccups with that as well :/

Thanks for your help!

-Austin