where for art thou, oh session attribute?

[rb]

In my think-client xmlrpc protocol to web2py svr code I am trying to
maintain data in the session global variable but it doesn't seem to
hold it. For example I first start by creating a document-form (tree
of datatables), cached in a data structure, as an attribute of
session. Here's the code:

from gluon.tools import Service

service = Service(globals())

def call():
return service()


@service.xmlrpc
def xrBeginDocFrm( frmName):
''' create table records and push xrInitRecord down to all tables.
'''
if session.frmz == None:
session.frmz = {}
if not session.frmz.has_key("frmName"):
session.frmz[frmName] = rna.DocFrm(db, frmName)

Then later I call the svr to get the table column definitions:

@service.xmlrpc
def xrGetColDefs(frmName, tblInstName):
tblInst = session.frmz[frmName].GetTblInst(tblInstName)
return tblInst.GetColDefs()

but I find that session does not have a frmz attribute. I can walk the
debuger through the svr code so I can see that the functions are
getting called, but when I inspect session it contains nothing.

I thought sessions stick around forever? I thought that session was
the place to keep my own attributes that will live on (between xmlrpc
calls).

Heeeeelp!

[mdipierro]

You cannot use sessions with xmlrpc because the xmlrpc protocol does
not use cookies to maintain persistence.

[rb]

Further, I added a :

session.hello = "hello, world"

in order to see if it just didn't want to hold onto MY data. No dice.
session.hello is gone on the next call as well.

---

[rb]

OMG. Then I'm screwed? Is there anyway for Web2py to keep state info
alive? Geez, I must be missing something. Do no web services keep
state info? Are all web services stateless?

Eek!

[mdipierro]

This is not a problem with the server but with the client.
You may try the cookie aware xmlrpc client proposed here:

http://code.activestate.com/recipes/501148/

Or you need to pass an extra key to the function calls and is that to
retrieve a session.

Massimo

[rb]

Well I need more than just cookies. I can't serialize the app
inbetween each call. I need a working process, which has active data
structures (database tables, dictionaires, etc).

Is there anyway for an app to talk out through web2py to a client?

I had (incorrectly) understood that sessions stick around and remain.
Even if a session object is cookized in between calls doesn't the docs
say that the session stuff remains? The manual says:

===
4.8 session
------------------
is another instance of the Storage class. Whatever is
stored into session for example:

session.myvariable="hello"

can be retrieved at a later time:

a=session.myvariable

as long as the code is executed within the same session by the same
user
(provided the user has not deleted session cookies and the session
did not
expire). Because session is a Storage object, trying to access an
attribute/key
that has not been set does not raise an exception, it returns None
instead.
===

Thus I can store myvariable into session and retrieve it at a later
time. What the manual _should_ say, if I understand you now, is that
the global variable "session" is pickled between each call into a
cookie and unserialized with each subsequent call. If your protocol,
eg xmlrpc does not use cookies then you can forget about sessions
being used to store info to be retrieved at a later time.

Argh! This is upsetting. I was thinking that session (and some
pythonic process) remained running in between calls. Argh.

Thx for the link, I'll check it out. (fingers crossed)

===

[mdipierro]

Once more, the issue is not with web2py. The issue is with your
client. It does not support cookies, hence it does not support
sessions.

If you use this client they should work:

http://fotinakis.com/blog/blog/2008/cookies-xmlrpc-and-ssl/

Massimo

[mdipierro]

Correction. You must use this client:
http://code.activestate.com/recipes/501148/
not this client:
http://fotinakis.com/blog/blog/2008/cookies-xmlrpc-and-ssl/



Massimo

[rb]

Thx kindly, Massimo. I'll check the links out... but...

>Once more, the issue is not with web2py. The issue is with your
client. It does not support cookies, hence it does not support
sessions.

Errr... no, that is not the issue that I'm pointing at. If "sessions
remain and do not expire" means that in between accesses the session
global variable is pickled to the disk and then later reconstituted
from the disk or else pickled into cookies and sent to the client,
then I have a problem with this definition. I don't think my data is
necessarily pickle-able.

I guess I misread/misunderstood paragraphs like:

The option masterapp=None, by default, tells web2py to try to
retrieve an
existing session for the application with name in request.application,
in the
running application.

to imply that there remained a *running process* that kept the session
variable (and hence my data) instantiated in between calls. I'm new to
this web app thing.

So, thx for the clarification. I'll either try to hack the code you
referenced or else I'll just end up using the web2py server as a data
server and keep the business logic in the thick client.

---
Rb

[mdipierro]

Normally when you visit a web page web2py creates a session, stores
the session in a file and associates the file to a session id. It
sends the session id back to the client in a cookie.

When the client makes other requests, the clients sends the cookie
back and the server can retrieve the proper session from the file. The
cookie is what allows the server to identify multiple requests as
coming from the same client.

This works in the same way with XMLRPC except that normally clients
ignore the cookie. So web2py does not have a way to recognize whether
two RPCs come form the same client or not so it assumes not. This is
because there is nothing about sessions in the specs of the XMLRPC
protocol.

One think you can do is the following:

from gluon.storage import Storage
rsession = cache.ram(request.client,lambda:Storage(),3600)

and use rsession instead of session. The different is that rsession
wil store session in ram (not disk), will expire (3600 seconds), and
will identify clients using their IP address instead of the cookie. If
you expect many IP to connect via XMLRPC this may cause a memory leak
since rsessions are never really deleted.


Massimo

[Yarko Tymciurak]

Ahem....


"Knock, knock!"  --- says the browser...
"Whose there?"  --- says the server...

If the request (browser)  identifies itself with information given it by the server as a continuation from a recent identification, then the normally stateless nature of the web get's state, established by the client identifying itself to the server....  typically this is done by cookies.

Client not saving session id?  Then client cannot establish it's identity --- eg. server has no session to "restore" --- this is a fresh (e.g. stateless) session.

See http://en.wikipedia.org/wiki/Session_management  for a bit of a conceptual overview.

[mdipierro]

anyway,
the poit raised by rb has some consequences we may want to think
about.

1) because clients ignore cookies this means there is a proliferation
of session files that are lost unless the xmlrpc server is told
explicitly to forget sessions. This means if one does not need
sessions one should use

def call():
session.forget()
return service()

2) should we provide a cookie aware xmlrpc cllient with web2py?

3) should we allow other mechanisms to id sessions? For example a uuid
in the URL?

On Aug 12, 2:23 am, Yarko Tymciurak <yark...@gmail.com> wrote:
> Ahem....
>
> "Knock, knock!"  --- says the browser...
> "Whose there?"  --- says the server...
>
> If the request (browser)  identifies itself with information given it by the
> server as a continuation from a recent identification, then the normally
> stateless nature of the web get's state, established by the client
> identifying itself to the server....  typically this is done by cookies.
>
> Client not saving session id?  Then client cannot establish it's identity
> --- eg. server has no session to "restore" --- this is a fresh (e.g.
> stateless) session.
>

> Seehttp://en.wikipedia.org/wiki/Session_management for a bit of a
> conceptual overview.

[rb]

A very good explanation. I think I get it now. In the bad ol' static
webpage days, a webserver's job was to figure out the mapping from the
requested url to the static file and then send that file back. Later,
dynamic webpages were created "on the fly" by processes which were fed
request parameters from incoming url requests: often these processess
were started and stop with each request.

I had thought that now in these modern days, with python, apache,
etc., that there was now some mechanism to keep the processes running,
and just feed requests to them and have them answer back their
responses, rather than starting and stopping a process (from a perl/
python/etc script). I think that J2EE offers this, no? I worked on a
(Tandem) mainframe a decade ago that offered this (albeit to the local
workstations: there was no WAN in their architecture back then).

That is, what you are describing is a system that maps urls to static
files or to python code in controllers et al, and uses a cookie to
transmit the key to the session file between the svr and the client.
Web2py pickles the session after every request and returns the cookie,
holding the id to the session, to the client - their is no stateful
process kept running or used.

I had thought it odd, at first, that the model was rendered with every
request (from the model files, processed in alphabetical order) in
order to provide the db global variable to the controllers, as I
thought that this was very expensive to do. But what do I know? I
haven't really tried to run any of this yet. Now I find that the
session info is saved and retrieved from a file (or else from a table
in the database) with every request. I guess it makes sense if its
purpose is to provide some "light" state info of the session. But I
may be abusing it, trying to use it to hold my datamodel/business-
logic-data-structures.

Hmmmm...

Maybe you could put #1 comment into the manual:

1) because clients ignore cookies this means there is a proliferation
of session files that are lost unless the xmlrpc server is told
explicitly to forget sessions. This means if one does not need
sessions one should use

def call():
session.forget()
return service()


2) should we provide a cookie aware xmlrpc cllient with web2py?

Err, xmlrpc is a light SOAP, no? SOAP does allow for stateful
connections, so why not xmlrpc as well?

3) should we allow other mechanisms to id sessions? For example a uuid
in the URL?

Is it possible for web2py to send the requests to another already-
running process and retrieve the answer? (Would this be like cgi?)

What about authentication then? Isn't a transmitted value/cookie
required in order to validate the credentials and permissions of the
request?

[rb]

Well the problem that I am having with all of this is the notion of
the svr *reconstituting* the state upon each request. Sure, the svr
must always have to identify the client and shuttle the request to the
desired functionality, but I didn't know the operating paradigm here
was that the "session data" was kept in a file, a key to that file was
created, and then the key was returned to the client, which must
thereafter send back this cookie in order to identify itself. This
action is fine for authentication but it is insufficient for
interacting with a complex model that is constructed from (many)
database tables and cached and organised in instantiated code objects.
I do not think it is economical to reconstruct the code objects upon
each and every request - thus my shock.

I stumbled into this situation because after coding up my client to
render the create/update/delete record model and view and the
gridtable for multiple-record selects, I realized that all my calls
back to the svr were basically DAL renderings of CRUD into data base
tables. I thought that it would be better to move the business logic
back into the svr. (E.g. some important table constraints are more
easily encoded into the python code than into the db, as I am starting
with SQLite, which does not support much in the way of table
constraints, but will want to move onto postgres later).

I guess I just have to maintain that all access to the database tables
goes thru the business logic, in order to maintain its integrity. I
can do this in the client if no other client of any kind can, at some
later time, gain access to the tables. This may sound odd but the last
company I worked for had this set up (in a major, internationally well
known, well used app). They used Crystal Reports to render their
reports via direct access to the tables and this caused no end of
problems (eg. some data was rendered on the fly by the business logic
when the tables are accessed from the app. Of course, this data does
not exist for the reports, because it does not exist in the database
tables. Oh the fun we had... )

===
Rb "web programming is haard."

On Aug 12, 12:23 am, Yarko Tymciurak <yark...@gmail.com> wrote:
> Ahem....
>
> "Knock, knock!"  --- says the browser...
> "Whose there?"  --- says the server...
>
> If the request (browser)  identifies itself with information given it by the
> server as a continuation from a recent identification, then the normally
> stateless nature of the web get's state, established by the client
> identifying itself to the server....  typically this is done by cookies.
>
> Client not saving session id?  Then client cannot establish it's identity
> --- eg. server has no session to "restore" --- this is a fresh (e.g.
> stateless) session.
>

> Seehttp://en.wikipedia.org/wiki/Session_management for a bit of a
> conceptual overview.

[desfrenes]

>2) should we provide a cookie aware xmlrpc cllient with web2py?
>
>3) should we allow other mechanisms to id sessions? For example a uuid
>in the URL?

These are very application-specific. Other services may require a
token in the method's parameters, other may require http auth, some
may need cookies etc... I wouldn't go beyond the xmlrpc spec on this
subject. Furthermore, a "good" service should be designed to work
with no state. State is the client's job, not the service, and this
makes your service more scalable. Then session.forget() should
probably be the default with return service().

[mdipierro]

My thought too.

[rb]

> Furthermore, a "good"  service should be designed to work
> with no state. State is the client's job, not the service, and this
> makes your service more scalable.

An n-tier (web) app has at least a database facility, business logic
facility, and presentation. Your comment above implies that all
business logic should exist in the client - and not in the svr,
because only trivial business logic is stateless. I don't think this
is necessarily true. The choices really boil down to three:

1) put stateful business logic in the svr, keep it active, and pass a
session token between client and svr (ie keep the state active in the
svr);

2) serialize the business logic state to disk (or cache) and then
throw away the state, and reconstitute the state upon each following
client request in the session (keep the state in the server but
unserialize and serialize it with each client request);

3) keep the business logic and its state (active) in the client and
only send stateless CRUD back to the svr.

I see advantages and problems with all three approaches.

[mdipierro]

If your client is willing to pass a state token this can be used to
retrieve the normal web2py session already by mimiking the cookie.
This can be done already but I need to write some instructions.

[desfrenes]

There have been countless flamewars on this subject... we need another
one here ;-)
As you guess, I think 3) is the way to go (although it doesn't have to
be limited to crud operations). I tend to trust Thomas Erl here
(http://www.soaprinciples.com/service_statelessness.asp).
Me and my team use json-rpc, which is pretty much the same as xmlrpc,
and most methods require a token, used only for auth ("can this client
use this method ?"), no state is kept on the server. We also consume
SOAP services that are not designed following the stateless principle
and it's a real pain in the... we even have to call methods in a
particular order just because they're not autonomous :-/
Of course you may have different needs but since they don't seem to be
covered by the xmlrpc protocol, you should probably address them in
your application. It shouldn't be too difficult to pass the cookie
value in the url, or as a parameter and then find the session data on
web2py side.
The problem is that there's no real standard for this, so any
implementation in web2py will probably remain web2py-specific. It's
probably ok, if it's not activated by default.

[Trollkarlen]

I had almost the same problem with flash.

Put this in the db.py to connect the session.

request.cookies[response.session_id_name]=request.vars.mysession_id
session.connect(request,response)

Then just send the session with pagename.html?
mysession_id=178648217648721...

/R

[weheh]

To add some levity to this important topic, rb is also, unfortunately,
confused about the meaning of "wherefore art thou, oh session
attribute?", which I believe him as asking where is the session
attribute, but in reality, it means, more approriately, "why and for
what purpose is the session attribute as it is?"

[rb]

<giggle>

That could be one way of interpreting it. But I was referring to an
attribute that I had added to session - thus it was my session
attribute. But with the very next request I found the session
attribute gone. Thus I whined "wherefore art thou, oh session
attribute."

Desfrenes points to an article, by Thomas Erl that begins:

"The management of excessive state information can compromise the
availability of a service and undermine its scalability potential.
Services are therefore ideally designed to remain stateful only when
required."

To which I have to respond: "of course." *Exessive* state information
would _never_ be a good thing. But what is "excessive" ? And he says
that services should remain stateful only when required...

Who is this guy? The Minister of the Obvious? <giggle>

The power and beauty of pushing the business logic out to the client
is that it leaves mostly stateless db access on the svr. Being
stateless makes for easier (dynamic) configuration of svr resources,
better scaling, and it makes maintaining the codebase simpler.

However, there is a cost to pushing the business logic out to the
client. Firstly it means that the client has to have more horsepower.
If the client is a pc then it's probably not a show stopper - but it
also means that deploying your app out to smaller netbooks, or
smartphones, is gonna have problems.

Albert Einstein said it was best to makes things as simple as possible
- but no simpler. Pushing the business logic out to the client can
lead to great problems down the road when query access (or some other
kind of access) is needed to be granted to other devices or other web
services that are acting as a client of our service. The point is that
access to the web app must come through the business logic, and this
constrains either the svr or the choice of clients.

I don't think there is any one answer for all situations, but I am
persuaded that the best choice is the one that moves data around the
least, and centralizes the activity that defines the app or service,
and allows for the leanest client. Of course, these are competing
goals, and my bias would be to prefer a #3 solution, as well, unless
specifics overruled.

>There have been countless flamewars on this subject... we need another one here ;-)

There is nothing new under the sun. (And "gee" does it have to erupt
into a flamewar?)

Further to other points made on this thread, I ask the question: what
about support for communication (rpc, or other) living processes? No
one would suggest that the web2py web server start adding statefulness
in order to provide such to specific applications, but why can't it be
made to support shuttling web traffic between distant clients and
local (stateful) processes?