Lazy initial session

[Anton Danilchenko]

(Sorry for Caps Lock)

I HAVE FOUND THAT YOU HAVE RESTORE SESSIONS IN THE MIDDLEWARE. BUT THIS CREATE A PROBLEM IN SOME CASES.

IN MY CASE - i HAVE DJANGO LIKE PROJECT STRUCTURE WHERE MODELS IMPLEMENTED IN THE EACH APPLICATION MODELS.PY FILE. MY FRAMEWORK INITIALIZE (LOAD) ALL MODELS FROM THE ALL APPLICATIONS. IF "USER" MODEL NOT LOADED - THAN I CAN'T RESTORE USER FROM SESSION - AND THIS IS A PROBLEM.

REAL SITUATION: I GO TO MY BLOG APPLICATION AND LOGIN. I HAVE STORED USER IN SESSION. RELOAD PAGE AND SEE ERROR (SEE ATTACHED FILE).

PLEASE IMPLEMENT LAZY LOADING (RESTORING) OF SESSION DATA ON THE FIRST ACCESS. THIS SAVE RESOURCES IF WE HAVE NOT ACCESS TO SESSION DICTIONARY IS SOME REQUESTS.

FOR EXAMPLE, IF I NEED GET SESSION INFORMATION IN MY BLOG APPLICATION - THAN WE NEED TO LOAD DATA FROM SESSION ON THIS STEP. NOT LOAD DATA ON SESSION INITIALIZATION, PLEASE!

THANKS!

[Anton Danilchenko]

Now you have call "__read_cookie" method on initialization. But we need do this on first call for data set or get data from datastore.

[David Underhill]

To avoid the KindError, you must import any model you store in your session in your main.py - otherwise the model may not be loaded when the session is loaded (e.g., if your instance is terminated and a new one is spun up).

Sessions are already lazily loaded; no data is loaded until the session is first accessed (e.g., a value is set or retrieved ... calling get_current_session() does not actually retrieve/load the data).

~ David

[Anton Danilchenko]

This is really not true. I have load my models before run wsgi script handler. My moels desined in separate files.

As we can see in backtrace - we have error because we restore data in session on creating wsgi handler.

In code below we have:

1) on first page load we:

  a) initial wsgi handler;

  b) import User model (we can load also other models from other applications, but we don't kwon about all models and can't hardcode imports)

  c) get user and store object in session

2) on second page load we:

  a) have "not implemented model" error because session don't know about User model. This model dinamically loaded AFTER initialization of wsgi handler

Try this code:

# file main.py - enter point for all user requests

from google.appengine.ext import webapp

from gaesessions import get_current_session

class Application(webapp.RequestHandler):

    def __init__(self):

        self.session = get_current_session()

        from models import User

    def get(self):

        # set user to session (only once)

        if "user" not in self.session:

            self.session["user"] = User.get_or_insert(email="te...@example.com")

        print self.session["user"]

if __name__ == "__main__":

    application = webapp.WSGIApplication([('/.*', Application)])

    run_wsgi_app(application)

SOLUTION: we need restore session data not on wsgi handler run, but ON FIRST dictionary lookup (add ne value or get value from session).

CURRENTLY we have loaded session data on wsgi handler initialization - but this is not good.

[Anton Danilchenko]

I understand. You initial fetch data from COOKIE, and in COOKIE saved my object (less than 10 Kbytes).

You need load data from COOKIE after first dictionary lookup. Thanks.

[David Underhill]

On Mon, Dec 13, 2010 at 11:39, Anton Danilchenko <an...@gaeframework.com> wrote:

class Application(webapp.RequestHandler):

    def __init__(self):

        self.session = get_current_session()

        from models import User

This line is your problem - you only import User inside a function call.  You need to move the import outside of any function call so that it is loaded when your main.py is loaded.

Otherwise, your code will crash when a new instance tries to load a session (and its user instance) without having ever seen the definition for the User model.

~ David

[David Underhill]

Loading data from the cookie should take an inconsequential amount of time (less than at most a few milliseconds).

Use AppStats on a production server (not dev_appserver) to get an idea of how long your requests are taking and what the performance issues are.

~ David

[Anton Danilchenko]

Can we restore data from cookies on the session first lookup? Why we have load sooki, if we maybe not need to work with sessions?

About models loading - I have show not real example. My models was load in specified place and added to global scope. App Engine understand where my lodels located and use this model classes.

I think that we need load COOKIES only on the first session lookup. Now we have load COOKIES on session class initialization - but this is not right place to do this. Please, thinking about this. Thanks!

[David Underhill]

On Mon, Dec 13, 2010 at 12:42, Anton Danilchenko <anton.da...@gmail.com> wrote:

Can we restore data from cookies on the session first lookup? Why we have load sooki, if we maybe not need to work with sessions?

If you don't need to work with sessions, then there will be no cookies to load.  If you are using sessions, there will be a cookie to load.  This does not affect end-user perception of performance at all (much less than a few milliseconds to load).

This is a pretty easy change, but won't save you anything (while adding complexity).  The patch to achieve this would look something like this:

http://pastebin.com/z18L5Kf2

Try AppStats and then decide what your performance bottlenecks are.  Loading the cookie isn't even going to register on the scale.

 

About models loading - I have show not real example. My models was load in specified place and added to global scope. App Engine understand where my lodels located and use this model classes.

You need to make sure the model is loaded before the middleware runs.  The only way to do this is import it in main.py's global scope.  If you're doing this, then you won't see a KindError.

If you still have issues, please provide sample code which demonstrates the problem.

~ David

[Anton Danilchenko]

I have disable using of Cookies by passing parameter like this:

app = SessionMiddleware(app, cookie_key=COOKIE_KEY, cookie_only_threshold=0)

and all work well/ Problem is in next. We have logged in to site, and user object stored to Cookie. We reload page and see on KindError. This is happens because Cookie values unpacked on Session class initialization, but session data was loaded on first access to this data in Session class object.

I believe that you understand me (if not - I am sorry for my bad explanation of this problem) and improve your library to do next - unpack data from cookies only on the first access to session data (like as you do this with session loading). If you do this - this is very help me, and I can back to enable cookie support.

Also I have a question. If I have store user information in cookies - than user can send me incorrect cookies. For example, I can logged in to site with user email "te...@example.com", but I send cookie where I specify that I have "ad...@site.com" email. This allow me access to site as administrator - and this is not good. This is correct?

Thank you for your time!

[David Underhill]

On Mon, Dec 13, 2010 at 14:57, Anton Danilchenko <anton.da...@gmail.com> wrote:

I have disable using of Cookies by passing parameter like this:

app = SessionMiddleware(app, cookie_key=COOKIE_KEY, cookie_only_threshold=0)

and all work well/ Problem is in next. We have logged in to site, and user object stored to Cookie. We reload page and see on KindError. This is happens because Cookie values unpacked on Session class initialization, but session data was loaded on first access to this data in Session class object.

I believe that you understand me (if not - I am sorry for my bad explanation of this problem) and improve your library to do next - unpack data from cookies only on the first access to session data (like as you do this with session loading). If you do this - this is very help me, and I can back to enable cookie support.

You can fix this problem with no changes to the library.  Import your model *before* you import/use gaesessions.  Then the KindError will be fixed.

Also I have a question. If I have store user information in cookies - than user can send me incorrect cookies. For example, I can logged in to site with user email "te...@example.com", but I send cookie where I specify that I have "ad...@site.com" email. This allow me access to site as administrator - and this is not good. This is correct?

Cookies are securely signed with HMAC-SHA256.  If the user modifies the cookie, the sessions library will know and will discard the cookies from that user.  A warning will also be logged.

 ~David

[Anton Danilchenko]

In my current project design I cant load models before I run wsgi application.

Sessions initialized not by me - but in the middleware. In this plcae automatically initialized your gaesession library, and in this place restored cookies. In this cookies I have stored object of my logged user. But in this place we dont know about loaded models and this is raise error.

As I propose above - we can not load cookies data (not restore data from cookies) on initializing of class Session. We can do loading of data from cookies similar with session data loading - on the first access to this data (first dicionary lookup) .

I have see that this should be fixed because we need lazy initialized loading of cookies data for user. For example, if I am not need to use cookies in curent request - than we not restore this data and we save resources of server. If I have do session lookup for get or set information - on this step you need restore data from cookie, load data from session (datastore and memcache) and I continue wotk well with loaded data.

We need only adding lazy initialization for cookies, as you already does with session data (stored in memcache and datastore).

[David Underhill]

http://pastebin.com/z18L5Kf2

If you must defer loading sessions, then apply the patch from the above link and cookies won't be read until they absolutely must be.

~ David

[Anton Danilchenko]

I not understand why you do not apply this patch to your project. Really we change loading data from all times to only one time if we need use this data. If we can't use this data - we not load any data from cookies and memcache and datastore.

What is my misunderstanding in this?

[David Underhill]

Before I apply the patch to the project I need to test it more thoroughly (mostly I need to tweak the test framework to properly deal with the fact that cookie info doesn't have to be loaded immediately).  It doesn't noticeably alter performance or fix a bug, so it isn't a high priority patch.

~ David

[Anton Danilchenko]

OK. Thanks. Now I have worked with sessons with disabled store data to Cookie. But I think that lazy loading of data from cookies is very good improvements and we can enable save data storing in cookies after you apply this patch.

[David Underhill]

You can enable storing data in cookies without the patch too.

Just import your model before the sessions library.

~ David

[Anton Danilchenko]

This is true. But I can't do this because I need initialize some parts of my framework and after this deside what models I need to load. For example, users can enable or disable some applications in configuration file. And I need load only models from specified applications. I can do this only after wsgi handler is start.

In any case - lazy loading is a good practice. If some users do use sessions in requests - this is continue work as now. But if user do not used sessions in some requests (for example, if I need show static page without additional information about user) - this is save server resources because I not load data from cookies, memcace and datastore.

Thank you! :-)

[David Underhill]

On Wed, Dec 15, 2010 at 04:02, Anton Danilchenko <anton.da...@gmail.com> wrote:

This is true. But I can't do this because I need initialize some parts of my framework and after this deside what models I need to load. For example, users can enable or disable some applications in configuration file. And I need load only models from specified applications. I can do this only after wsgi handler is start.

In any case - lazy loading is a good practice. If some users do use sessions in requests - this is continue work as now. But if user do not used sessions in some requests (for example, if I need show static page without additional information about user) - this is save server resources because I not load data from cookies,

True, it certainly doesn't hurt.  I'll add this later.

 

memcace and datastore.

We already lazy load memcache/datastore - it is big hit not too when users don't need the session.

~ David

 

Thank you! :-)

[Anton Danilchenko]

You can complete this change in this year? Thanks!