How to store session data server side?

[Anders Wegge]

 As I read the documentation for SignedCookieSessionFactory, the data stored in the session is not encrypted. So storing a CSRF token in the session Cookie is not a good option. Pyramid_beaker seem to have been deprecated with release 1.5, so which options are the best  for a site with very few actions requiring CSRF and other session data. My first thought is to pickle the session data, and storing them in a blob in the user database, but if there is something simpler and/or more elegangt available, I'd like to hear the alternatives.

[Jonathan Vanasco]

You can still use pyramid_beaker.  It works fine under pyramid 1.5 .  

There are a few heated discussions in the archives over it.  It's a sore point for many.  I could be wrong on this, but last I checked it's not so much "deprecated" as it is "off the radar".    A lot of us still use it for server side sessions.  If it ever breaks there will be patches or a migration plan to something else.  

One of the better candidates right now is pyramid_redis_sessions ; the only downside of it is that it only supports redis as a backend.  

for our needs, we needed a single package that could handle both a memcached/redis for production , but a pickled file storage for development.  so we're still on beaker.

[Randall Leeds]

I could be wrong, and would love to be corrected, but storing the CSRF token in a session cookie doesn't seem problematic to me since the cookie should only be readable by your origin.

It would only become a concern of you started allowing cross origin requests with credentials.

On Apr 23, 2014 8:09 AM, "Anders Wegge" <awe...@gmail.com> wrote:

 As I read the documentation for SignedCookieSessionFactory, the data stored in the session is not encrypted. So storing a CSRF token in the session Cookie is not a good option. Pyramid_beaker seem to have been deprecated with release 1.5, so which options are the best  for a site with very few actions requiring CSRF and other session data. My first thought is to pickle the session data, and storing them in a blob in the user database, but if there is something simpler and/or more elegangt available, I'd like to hear the alternatives.

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To post to this group, send email to pylons-...@googlegroups.com.
Visit this group at http://groups.google.com/group/pylons-discuss.
For more options, visit https://groups.google.com/d/optout.

[Chris McDonough]

On 04/23/2014 12:06 PM, Jonathan Vanasco wrote:
> You can still use pyramid_beaker. It works fine under pyramid 1.5 .
>
> There are a few heated discussions in the archives over it. It's a sore
> point for many. I could be wrong on this, but last I checked it's not
> so much "deprecated" as it is "off the radar". A lot of us still use
> it for server side sessions. If it ever breaks there will be patches or
> a migration plan to something else.

Right, it's just not maintained. If someone wants to maintain it,
they're free to pick it up. I wrote the original bindings but I
can't/won't maintain it anymore. The Beaker code itself has no current
maintainer, and I personally won't maintain bindings to something that
has no current maintainer. If someone else wants to do that, they can;
they'll also then become defacto Beaker maintainer, AFAICT.

> One of the better candidates right now is pyramid_redis_sessions ; the
> only downside of it is that it only supports redis as a backend.

This is what I use when I need server-side sessions. As far as I can
tell, the only folks for whom the Redis-only-backendedness might be a
problem are folks that run servers on Windows.

- C

[Jonathan Vanasco]

On Wednesday, April 23, 2014 12:56:11 PM UTC-4, Chris McDonough wrote:

Right, it's just not maintained.   If someone wants to maintain it,
they're free to pick it up.  I wrote the original bindings but I
can't/won't maintain it anymore.  The Beaker code itself has no current
maintainer, and I personally won't maintain bindings to something that
has no current maintainer.  If someone else wants to do that, they can;
they'll also then become defacto Beaker maintainer, AFAICT.

Beaker is a mess.

If anyone were to maintain, it would be migrating the caching-only to dogpile and using pyramid's cookie functionality for everything else.  I looked into it, but it was too much work for a "useful" thing.  If it were "necessary", that's a different story.

This is what I use when I need server-side sessions.  As far as I can
tell, the only folks for whom the Redis-only-backendedness might be a
problem are folks that run servers on Windows.

We're Mac and Linux; it's a problem for us.  We only "rely" on redis in production; not staging or dev (sometimes present for messaging, sometimes not).  beaker lets us use file-based sessions on our dev systems.  I'm okay with using different backend datastores in each environment, but I don't like the idea of different sessioning systems in each one.  Doing stuff like that has caused too many issues when trying to pinpoint a bug in the past.

 

[Chris Rossi]

"sudo apt-get install redis-server" is pretty easy.  Then you have a service on localhost listening to the default port, there's pretty much zero configuration required.

Chris

[Randall Leeds]

I don't think I explained this well.

Storing sensitive data in the session cookie is only a problem when the data is sensitive to inspection by your _legitimate_ users. For instance, if you were making a gambling game you wouldn't want to store information about the game state that isn't visible on the gambling table, e.g. the dealer's hand. With that information, the player could cheat.

The CSRF is often rendered in your forms as a hidden field, which means it's always available by simple inspection of the page source. It's not sensitive in the same way.

The CSRF protection comes from ensuring that a cross site request cannot obtain an CSRF token and cause a subsequent request with the corresponding session cookie. Without cross-origin credentialed requests the attacker cannot send the session cookie.

Even if an attacker can't decrypt the cookie, so long as they can send it they can pass the CSRF check. If your session cookie can be hijacked it's a problem. To prevent that you need guard against cross-site scripting vulnerabilities (sanitize user-submitted HTML and such) and use SSL.

[Bert JW Regeer]

Hey Anders,

Storing the CSRF token in the cookie is not an issue, unless an attacker is able to read out the cookie and thus gain the CSRF token, they won’t be able to get access to it at all, even if you stored the CSRF token server side, if an attacker was able to get access to the cookie (using cross site scripting or code injection or a variety of other techniques…) even if you stored the CSRF token itself server side they would have a unique token that is tied to that server side session storage, and thus be able to pass any CSRF checks anyway (since they can read your cookies, they can also read the page itself and retrieve the CSRF token, since it is stored in a form that is submitted to the server).

Using the SignedCookieSessionFactory is perfectly safe and sane when all you are doing is storing limited data such as the CSRF token.

Bert

[Jonathan Vanasco]

On Wednesday, April 23, 2014 2:23:03 PM UTC-4, Chris Rossi wrote:

"sudo apt-get install redis-server" is pretty easy.  Then you have a service on localhost listening to the default port, there's pretty much zero configuration required.

It's a breeze to run and manage for sure; especially on Production linux machines.

It's added overhead when managing our staging machines (which actually do have redis already installed, but tuned for other systems.).  For local development on macs, it's on most machines -- but we don't want to rely on it when onboarding new devs, setting up new machines, working with contractors, etc.  There's just too much setup and room for error with so many moving parts -- especially when we have people who work on the front-end stuff only.  We had to draw a line in the sand somewhere , and requiring redis on machines was just too much.  not everyone is a full-stack developer (or wants to be one); requiring too many server-side apps and general know-how limits the pool of resources we can bring in.

The beauty of Beaker (and dogpile) is that the configured backend only handles where serialized data is tucked away.  we haven't had an issue switching between backend stores.  when testing other "swappable" components, we've had issues pop up in one environment , but not another; often do to how different objects are serialized.  we're just not at a place to take the risk of encountering this between systems or having to manage a second set of tests and edge-cases tailored to our integration against each plugin.  

[wilk]

Chris,

Somes times ago you said that we can use SignedCookieSessionFactory with
a custom serializer to just pass an ID and serialize/deserialize to the
backend that we want.
For loads, it's ok, the parameter is the ID, we can retrieve our data
like we want.
But for dumps the parameters are a tupple, the last item is the dict of
the session. If we want to keep the same ID we must find it on this
dict, yes ? The documentation of this parameter is not clear for me...

Could you give an example of a serializer object wich will record the
session on a dict in memory for example ?

Don't know if my question is clear...

--
William

[Michael Merickel]

On Wed, Apr 23, 2014 at 3:59 PM, wilk <wi...@flibuste.net> wrote:

Could you give an example of a serializer object wich will record the
session on a dict in memory for example ?

https://gist.github.com/mmerickel/8035611

Unfortunately there's one little quirk that you can't reuse the session id, so it changes per request, which may be a problem if you're using a session in combination with concurrent ajax requests (a problem with any blob-based session storage really). This could be fixed by using get_current_request() or by modifying the Pyramid api slightly to pass the request into the serializer.

Nice part about this is that the session cookie is securely tracked by Pyramid and you're only responsible for cleaning up old sessions in the backend.

[Chris McDonough]

On 04/23/2014 04:17 PM, Jonathan Vanasco wrote:
>
>
> On Wednesday, April 23, 2014 2:23:03 PM UTC-4, Chris Rossi wrote:
>
> "sudo apt-get install redis-server" is pretty easy. Then you have a
> service on localhost listening to the default port, there's pretty
> much zero configuration required.
>
>
> It's a breeze to run and manage for sure; especially on Production linux
> machines.
>
> It's added overhead when managing our staging machines (which actually
> do have redis already installed, but tuned for other systems.). For
> local development on macs, it's on most machines -- but we don't want to
> rely on it when onboarding new devs, setting up new machines, working
> with contractors, etc. There's just too much setup and room for error
> with so many moving parts -- especially when we have people who work on
> the front-end stuff only. We had to draw a line in the sand somewhere ,
> and requiring redis on machines was just too much. not everyone is a
> full-stack developer (or wants to be one); requiring too many
> server-side apps and general know-how limits the pool of resources we
> can bring in.

<cough> buildout.

Seriously, you can't beat it for supplying new hires and customers with
a single command that can both install and update an environment. It
can install Python packages, but it can also install "CMMI"
(configure/make/make install) packages and run command line scripts and
so on. I couldn't imagine running a consulting project without it.
Well, actually, I could, because I've been on consulting projects that
refuse to use it, and they are always horrible.

- C

[wilk]

On 23-04-2014, Michael Merickel wrote:
> --001a11c3f51a5ce90d04f7bc3072
> Content-Type: text/plain; charset=UTF-8

>
> On Wed, Apr 23, 2014 at 3:59 PM, wilk <wi...@flibuste.net> wrote:
>
>> Could you give an example of a serializer object wich will record the
>> session on a dict in memory for example ?
>>
>
> https://gist.github.com/mmerickel/8035611
>
> Unfortunately there's one little quirk that you can't reuse the session
> id, so it changes per request, which may be a problem if you're using a
> session in combination with concurrent ajax requests (a problem with any
> blob-based session storage really). This could be fixed by using
> get_current_request() or by modifying the Pyramid api slightly to pass the
> request into the serializer.
>

Thanks, it's what i thought. I try a version to keep the ID. We also
have the 'accessed' parameter which will help to clean the old sessions.

https://gist.github.com/flibustenet/11248284

Do you think it can be an interest to make public the structure of
`appstruct` and probably add the `request` to dumps and loads ?

I have an interest of a simple memory session like that for little web
services, I can try to submit a patch.

> Nice part about this is that the session cookie is securely tracked by
> Pyramid and you're only responsible for cleaning up old sessions in the
> backend.

Yes, it was a good idea.

--
William

[Anders Wegge]

In the classic meaning of CSRF, you are right. But if javascript from a malicious site can get access to all cookies in the browser, it would be trivially simple to construct a XmlHttpRequest, that contain the correct CSRF token. While most browsers are sandboxing data, I do not want to rely on that.

[Jonathan Vanasco]

FWIW, I handle my user interactions like this:

there's a 'normal' pyramid beaker session on http sessions used to maintain login state and display data.  ( request.session )

there's a secondary https only session , provided by a quick library i put together ( request.session_https )  pyramid_subscribers_beaker_https_session 

the https session records, on the server side, details about the login ( the time; the source [by form, by auto-login, by facebook], etc )

different site functions require different login credentials.  for example, updating account info requires a "Form" login refresh of 5 minutes.  

anything in "/account" is on https.  all ajax requests that deal with user data or "write" requests, are piped through to the https server via a "porthole".   https still uses it's own csrf too.

[Randall Leeds]

On Apr 24, 2014 7:39 AM, "Anders Wegge" <awe...@gmail.com> wrote:
>
> In the classic meaning of CSRF, you are right. But if javascript from a malicious site can get access to all cookies in the browser, it would be trivially simple to construct a XmlHttpRequest, that contain the correct CSRF token. While most browsers are sandboxing data, I do not want to rely on that.

At that point the browser is totally broken. I would think hard about whether this is really in your threat model.

[Chris Rossi]

That's what I was thinking.  If that's broken in browsers then pretty much the whole www is broken.  At a certain point you have to trust something to work.

Chris

[Mike Orr]

On Tue, Apr 22, 2014 at 1:16 PM, Anders Wegge <awe...@gmail.com> wrote:
> As I read the documentation for SignedCookieSessionFactory, the data stored
> in the session is not encrypted. So storing a CSRF token in the session
> Cookie is not a good option. Pyramid_beaker seem to have been deprecated
> with release 1.5, so which options are the best for a site with very few
> actions requiring CSRF and other session data.

I've used Beaker with Pylons for several years and pyramid_beaker for
several months without any problems. When I heard about the
deprecation I was unhappy, but later decided to try
pyramid_redis_sessions anyway to evaluate Redis for more widespread
use (caching, usage stats, all data). So I compiled Redis and set up
pyramid_redis_sessions, and it's working flawlessly.

The 'pyramid_[session]' packages are all pretty small; most of the
code is in the backend packages. The reason Beaker's creator stopped
supporting it is it's old, messy, hard-to-maintain code. It was
originally ported from Perl::Mason via Myghty circa 2005. Since then
we've learned a lot more about the virtues of modularity and what
contemporary web applications need. Dogpile is a "clean" renovation of
Beaker's storage system. It's really only one of Beaker's backends, so
it's not a complete replacement of Beaker.

There are now three fundamental questions for session-adapter
developers. Do we want to replicate Beaker with a full variety of
backends, including Dogpile and database and others? Or do the
'pyramid_[session]' packages sufficiently replace the Beaker frontend,
and we just need to make 'pyramid_[session]' packages for the full
variety of backends? Or move away from sessions?

In Beaker's time, everybody used sessions if they had inter-request
temporary data. Now some people prefer to store it all in client-side
storage (cookies, Javascript variables, HTML 5 storage) to avoid the
scaling issues of server-side sessions. Others maintain a session ID
but store all the data in ordinary database tables. (That gets complex
if you need 1:many tables.)
Others do other things I don't understand.

I use server-side sessions because (1) I have large pieces of data
like all the result IDs from a search, and (2) sometimes i don't want
the client to see the data directly.

The main usability difference between Beaker and the
'pyramid_[session]' packages is that Beaker has a unified
configuration API for its backends. To switch backends you just change
one INI setting. But with the 'pyramid_[session]' packages you have
to change your includes and dependencies, and each one has different
settings variables. I had to switch from one requirements file to
several, to allow people to use various session packages without
installing all of them.

So that would be the advantage of a unified Beaker replacement: a
unified configuration. But it's not an overwhelming advantage, because
the 'pyramid_[sessions]' packages also work pretty well, and have the
advantage of being independent.

So ultimately we need either a Beaker replacement or packages for file
sessions and SQL-database sessions. I guess Memchached is obsolete now
that Redis can do the same? I'm also exploring some alternatives for
Redis, such as putting the session in a Redis hash rather than a Redis
string, with the values as either pickle or JSON. That would make it
easier to manage sessions in the Redis CLI rather than just having
opaque Python-specific pickles.

I tried a bit to make a 'pyramid_dogpile', and to store data in
database tables, and to modify Michael's module to keep the same
session ID, and other things, but they're all so complicated. Pyramid
1.5 makes it a little bit easier because it includes some signing
stuff that previously each package had to maintain.
'pyramid_redis_sessions' hasn't yet caught up to that code yet.
(Although it's really the same code, just in a different place.)

I also wish Pyramid had the foundations for generating session IDs and
saving them in cookies and keeping the same ID, because those are the
hardest parts of writing a backend. Again, Beaker has it all, but it's
hard to extract code from Beaker because it's so messy.

I also ran into an issue the Beaker puts the session ID in the session
as an attribute (session.id) but pyramid_redis_sessions doesn't. That
caused me problems because I was keeping the session IDs in my access
log to count the number of sessions per month, their lengths, and
entry and exit pages. I couldn't figure out how to do that with
pyramid_redis_sessions. Finally somebody suggested putting a different
random ID in the session, independent of the session ID, and that
worked.

[Chris McDonough]

On 04/26/2014 03:58 PM, Mike Orr wrote:
> On Tue, Apr 22, 2014 at 1:16 PM, Anders Wegge <awe...@gmail.com> wrote:
>> As I read the documentation for SignedCookieSessionFactory, the data stored
>> in the session is not encrypted. So storing a CSRF token in the session
>> Cookie is not a good option. Pyramid_beaker seem to have been deprecated
>> with release 1.5, so which options are the best for a site with very few
>> actions requiring CSRF and other session data.
>

> There are now three fundamental questions for session-adapter
> developers. Do we want to replicate Beaker with a full variety of
> backends, including Dogpile and database and others? Or do the
> 'pyramid_[session]' packages sufficiently replace the Beaker frontend,
> and we just need to make 'pyramid_[session]' packages for the full
> variety of backends? Or move away from sessions?

I'm only motivated to write and maintain session bindings packages when
a customer pays me to do so (eg. "we want to store all of our data in
X", often because they haven't really yet figured out that session data
is not much like their other data).

I wouldn't personally create a Beaker replacement (meaning a package
which is itself a framework for plugging in different backends), because
I use Pyramid pretty much exclusively, and most of the contracts about
what makes up a session already exist in Pyramid, and thus I'd just make
a pyramid_foo if I had to. If some of the pyramid_foo code was more
generally usable, it might go into a separate package (ala Dogpile), but
I wouldn't make an omnibus package that had as its mission "sessions".

- C

[wilk]

Hi,

On one hand to write a pyramid_session_foo is not very difficult, on the
other hand beaker seems to be impossible to maintain and nobody seems to
want to do any pyramid_session_foo. For newcomer it's a little bit
ambiguous to see that the recommended session manager are redis or an
unmaintained beaker.

I'd like to see how could look pyramid_redis_session if it use
SignedCookieSessionFactory. I mean, i'd like to see examples of custom
session with the help of this new class.

For example, what is better ? to write a serializer like
https://gist.github.com/mmerickel/8035611 (but we have a little problem
with the ID)

or to make a new factory which use a signedcookie like
https://github.com/Pylons/pyramid/pull/1142
def FileBasedSessionFactory(base_path, secret, cookie_name, **kwargs):
session_cookie_factory = SignedCookieSessionFactory(secret, cookie_name, **kwargs)

class FileSession(dict):
def __init__(self, request):
self.cookie = session_cookie_factory(request)
...

thanks for your advices

--
William

[Jonathan Vanasco]

1. Like I mentioned above, but not very clearly, the big reason why I dislike the pyramid_session_[foo] approach, is that the only standardization is how it adapts to the ISession interface. There's no standardization on the (de)serialization interface.  With the Beaker model, almost everything is the same -- if you switch backend storage, you don't run into any surprises.  There's also a general standardization on the configuration settings.  You can swap between backends easily and without surprise.

With the pyramid_session_[foo] approach, although the developer interface is similar, under-the-hood it's quite different.  We ran into issues while testing with how objects were able to be (de)serialized , and switching from one backend to another meant needing an entirely different approach for configuration.  if you're willing to buy-in to a single sessioning package for good, it's fine. If you need to run different packages in different environments, or might change in the future, it's a hassle.

2. I still really think that ISession should support an official `session_id` attribute

3. chris- buildout is great. i've used it many times.  but as you deal with larger and more varied organizations, it loses it's utility as "The Solution".  if everyone involved in a project is decently versed in Python and has some backend knowledge -- great.  But once you start adding in dedicated HTML/CSS/JS people or graphic designers who need to update assets and can barely use a GUI version tracking app... you end spending all the savings in "setup" on troubleshooting individual people's machines.  being able to run the app with as few services as possible is , in a word, "heavenly".

[Chris McDonough]

On 04/28/2014 11:04 AM, Jonathan Vanasco wrote:
> 1. Like I mentioned above, but not very clearly, the big reason why I
> dislike the pyramid_session_[foo] approach, is that the only
> standardization is how it adapts to the ISession interface. There's no
> standardization on the (de)serialization interface. With the Beaker
> model, almost everything is the same -- if you switch backend storage,
> you don't run into any surprises. There's also a general
> standardization on the configuration settings. You can swap between
> backends easily and without surprise.
>
> With the pyramid_session_[foo] approach, although the developer
> interface is similar, under-the-hood it's quite different. We ran into
> issues while testing with how objects were able to be (de)serialized ,
> and switching from one backend to another meant needing an entirely
> different approach for configuration. if you're willing to buy-in to a
> single sessioning package for good, it's fine. If you need to run
> different packages in different environments, or might change in the
> future, it's a hassle.

There actually is standardization on serialization and deserialization.
The ISession interface docstring:

class ISession(IDict):
""" An interface representing a session (a web session object,
usually accessed via ``request.session``.

Keys and values of a session must be pickleable.
"""

As the class docstring indicates, the only contstraints on keys and
values are that they must be pickleable. If some sessioning
implementation does not provide this feature, it doesn't meet the
ISession interface (which doesn't mean it's not useful, it just means
it's not really in compliance). This would be a bug in an
implementation that aims to provide compliance. Were you running into
something that could not be pickled, or?

>
> 2. I still really think that ISession should support an official
> `session_id` attribute

The reason there isn't a "session_id" API is because not all sessions
have an ID. For example, the cookie session stuff in the core doesn't
have (nor need) an ID, because there's no storage to need to look the ID
up in to retrieve the session data. Meanwhile, it's pretty simple to
put a random value into the session to act as an application-specific
ID. That's the rationale anyway.

> 3. chris- buildout is great. i've used it many times. but as you deal
> with larger and more varied organizations, it loses it's utility as "The
> Solution". if everyone involved in a project is decently versed in
> Python and has some backend knowledge -- great. But once you start
> adding in dedicated HTML/CSS/JS people or graphic designers who need to
> update assets and can barely use a GUI version tracking app... you end
> spending all the savings in "setup" on troubleshooting individual
> people's machines. being able to run the app with as few services as
> possible is , in a word, "heavenly".

Yeah, I dunno. We use buildout quite often to deliver a locally-running
system to nontechnical users and graphic designers. But we're also
committed to making sure that we build most everything from source when
it matters (e.g. sometimes redis, postgres, etc), so over time it
becomes less and less likely that the build won't work. I realize this
is a committment that not everyone is willing to make.

- C

[Jonathan Vanasco]

On Monday, April 28, 2014 1:11:17 PM UTC-4, Chris McDonough wrote:

As the class docstring indicates, the only contstraints on keys and
values are that they must be pickleable.  If some sessioning
implementation does not provide this feature, it doesn't meet the
ISession interface (which doesn't mean it's not useful, it just means
it's not really in compliance).  This would be a bug in an
implementation that aims to provide compliance.  Were you running into
something that could not be pickled, or?

I don't remember what we were using, but some of the backend drivers we had at one point weren't using `Pickle`

The reason there isn't a "session_id" API is because not all sessions
have an ID.  For example, the cookie session stuff in the core doesn't
have (nor need) an ID, because there's no storage to need to look the ID
up in to retrieve the session data.  Meanwhile, it's pretty simple to
put a random value into the session to act as an application-specific
ID.  That's the rationale anyway.

i strongly disagree with the rationale.

client-side sessions don't have an id, but server-side sessions require it.  because pyramid doesn't have an official interface to support this, all server side session libraries need to implement this themselves -- somehow -- to support it. I stress "somehow".  There's no way to ensure or expect consistency in this across libraries. If you look at how different session providers in the ecosystem have dealt with this, there is little uniformity in approach.  I've seen `session_id`, `_session_id`, `sessionID` and a few other variations -- both as attributes and internal dict values.  If ISession had an official `session_id` attribute, then people developing server side libraries would adapt to that; there would be uniformity and portability.   Those using client-side sessions would never use or expect a session_id , and not be inconvenienced.

Yeah, I dunno.  We use buildout quite often to deliver a locally-running
system to nontechnical users and graphic designers.  But we're also
committed to making sure that we build most everything from source when
it matters (e.g. sometimes redis, postgres, etc), so over time it
becomes less and less likely that the build won't work.  I realize this
is a committment that not everyone is willing to make.

That's a commitment everyone I know would love to make, but can't afford the resources to act on.

 

[Michael Merickel]

On Mon, Apr 28, 2014 at 3:05 PM, Jonathan Vanasco <jona...@findmeon.com> wrote:

I don't remember what we were using, but some of the backend drivers we had at one point weren't using `Pickle`

Pyramid has an ISession interface and that is the point of pluggability of sessions. Due to this, there really isn't an explicit need for some multi-backend library. Beaker has one because Pylons and other frameworks did not define what a session should be in the way that Pyramid does. Bindings can use the SignedCookieSessionFactory, or the underlying and more useful webob.cookies.SignedCookieProfile, to handle the actual cookie management (tracking the id), at which point they basically just need to implement a dict that tracks changes and persists them to their backend. And yes, Pyramid requires they support pickleable stuff if you want it to work with arbitrary Pyramid apps right now.

 

client-side sessions don't have an id, but server-side sessions require it.  because pyramid doesn't have an official interface to support this, all server side session libraries need to implement this themselves -- somehow -- to support it. I stress "somehow".  There's no way to ensure or expect consistency in this across libraries. If you look at how different session providers in the ecosystem have dealt with this, there is little uniformity in approach.  I've seen `session_id`, `_session_id`, `sessionID` and a few other variations -- both as attributes and internal dict values.  If ISession had an official `session_id` attribute, then people developing server side libraries would adapt to that; there would be uniformity and portability.   Those using client-side sessions would never use or expect a session_id , and not be inconvenienced.

Why do clients (developers using request.session) need the session id? How is this not an implementation detail of a single session factory? Are you dual-purposing this id for things? Do you expect other session backends to reuse this id somehow?

[Chris McDonough]

On 04/28/2014 04:05 PM, Jonathan Vanasco wrote:
>
>
> On Monday, April 28, 2014 1:11:17 PM UTC-4, Chris McDonough wrote:
>
> As the class docstring indicates, the only contstraints on keys and
> values are that they must be pickleable. If some sessioning
> implementation does not provide this feature, it doesn't meet the
> ISession interface (which doesn't mean it's not useful, it just means
> it's not really in compliance). This would be a bug in an
> implementation that aims to provide compliance. Were you running into
> something that could not be pickled, or?
>
>
> I don't remember what we were using, but some of the backend drivers we
> had at one point weren't using `Pickle`

OK, these drivers did not actually implement ISession then.

>
> The reason there isn't a "session_id" API is because not all sessions
> have an ID. For example, the cookie session stuff in the core doesn't
> have (nor need) an ID, because there's no storage to need to look
> the ID
> up in to retrieve the session data. Meanwhile, it's pretty simple to
> put a random value into the session to act as an application-specific
> ID. That's the rationale anyway.
>
>
> i strongly disagree with the rationale.
>
> client-side sessions don't have an id, but server-side sessions require
> it. because pyramid doesn't have an official interface to support this,
> all server side session libraries need to implement this themselves --
> somehow -- to support it. I stress "somehow". There's no way to ensure
> or expect consistency in this across libraries. If you look at how
> different session providers in the ecosystem have dealt with this, there
> is little uniformity in approach. I've seen `session_id`,
> `_session_id`, `sessionID` and a few other variations -- both as
> attributes and internal dict values. If ISession had an official
> `session_id` attribute, then people developing server side libraries
> would adapt to that; there would be uniformity and portability. Those
> using client-side sessions would never use or expect a session_id , and
> not be inconvenienced.

IMO, the point of an interface is to provide an abstract API which folks
can meet to provide alternate implementations. If server-side session
implementations had a session id, but client-side session
implementations did not, it would mean that *deployers* of an
application could not switch out the sessioning implementation from one
to the other and still expect the parts of their applications which rely
on sessions to work, which is AFAICT exactly what you're objecting to
now. The only difference between this scenario and the one you're
objecting to is that you started relying on something *outside* the
interface, and when you changed to using something that actually
complied with it, you got hosed.

In the meantime, AFAICT, it's reasonably simple to put a random id (or
something composed with values unique to the originating request, e.g.
'%s-%s' % (id(request), time.time()) or something) as a value in the
session dict. This would make the application run under any sessioning
implementation. This makes a lot more sense to me than having two
separate ISession interfaces, one with an outside-the-session id, and
one without.

If you're saying that two different applications written by two people
that are unaware of each other need access to the *same* session id
value, then, yes, this doesn't work, unless it's just by a-priori
agreed-upon convention. But I am having trouble envisioning one
application relying on another's use of the same session value to
represent an "id". Can you describe a case for this? Note that I don't
mean two applications that *you* write, because of course you can use a
convention for that, but two applications that must work together that
are written by two totally separate people who have never met each other
or had an opportunity to agree on a shared key name.

- C

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/28/2014 06:07 PM, Chris McDonough wrote:
> written by two totally separate people who have never met each other
> or had an opportunity to agree on a shared key name.

They need to agree on more than the name: the type / range of the value
matter, too (e.g., sequence-generated int's vs stringified UUIDs).


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 tse...@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlNe3eAACgkQ+gerLs4ltQ44FwCgzR+WaHiNXnpk0AprajnXckm8
MR0An1VJocE4BtzyqzmLioInBC4GGMEx
=GpDY
-----END PGP SIGNATURE-----

[Bert JW Regeer]

So this discussion has been going on for quite some time, and each time I saw this email come by I figured there has to be a better way than to continue going on whether there should be an session_id on the session or not.

I personally don’t think there should be, and it should be implementation defined, so to that end I have actually ended up writing pyramid_pluggable_session. It’s really simple, it even takes the same parameters as the default SignedCookieSessionFactory(), it couldn’t be simpler … ;-)

So here you can find the code:

https://github.com/usingnamespace/pyramid_pluggable_session

Now, this is 0.0.0dev, that is no checking what so ever, no tests, nothing, and it only comes with a single pluggable interface, and it uses local memory without threading support so it’s dangerous, but sometimes I like to live on the edge ;-)

I hear you say, well how do I use it… perfect, stick this in your .ini:

pluggable_session.secret = ThisShouldBeSecret
pluggable_session.plug = pyramid_pluggable_session.memory.MemorySessionPlug

And add:

pyramid_pluggable_session to your pyramid_includes or do config.include(‘pyramid_pluggable_session’).

Plugging is simple:

https://github.com/usingnamespace/pyramid_pluggable_session/blob/master/pyramid_pluggable_session/memory.py

Now, the backend in my case simply needs to store the opaque object, the backend shouldn’t care what the data being stored is at all, which even in the case of using a database is perfect, no exploding amount of tables required, cause normalisation doesn’t come into play here.

Now each loads() gets access to the session object (the only thing initialised at that time is the session_id though …) and the request (that is where you can stash your connections to redis/database and get access to the registry).

Each dumps() gets access to the session object, this time all the required ISession variables are available, such as .created, so you can store that in your DB for expiration purposes, along with the session_id of course, and then the request object, and the session_data.

In my implementation the session_data is serialized using Pickle, it is then Signed. This means that even data coming from the backend is verified that it hasn’t been tampered with (or more likely, no data corruption has happened).

The only thing sent to the client is a cookie containing the session id. Now the session id is 20 bytes of random that has been hexlified. The cookie data is serialised/signed using the SignedCookieHelper in WebOb. So if someone wanted to tamper with the cookie (to change the session ID for example) they would also need to know your secret…

Now go forth and enjoy. I’ll accept pull requests for file based storage, redis/mongo/db based storage, and other stuff like that. Or even better, create pyramid_sessplug_<name>, so that people that want to use certain ones can use them.

Bert JW Regeer

[Bert JW Regeer]

Oh, forgot to include this link, here is a very simple starter app that uses this session to show how many times you have visited the page:

https://github.com/bertjwregeer/pyramid_test_pluggable_session

It uses the dangerous memory session plug though …

Bert

[Mike Orr]

On Mon, Apr 28, 2014 at 3:07 PM, Chris McDonough <chr...@plope.com> wrote:
> On 04/28/2014 04:05 PM, Jonathan Vanasco wrote:
>>
> IMO, the point of an interface is to provide an abstract API which folks can
> meet to provide alternate implementations. If server-side session
> implementations had a session id, but client-side session implementations
> did not, it would mean that *deployers* of an application could not switch
> out the sessioning implementation from one to the other and still expect the
> parts of their applications which rely on sessions to work, which is AFAICT
> exactly what you're objecting to now.

We don't need two interfaces. We just need an interface that says
'session.id' or 'session.get_session_id()' should return the session
ID string, or None if it doesn't support session IDs or doesn't wish
to expose it. Presto, everybody's problem is solved.

> The only difference between this
> scenario and the one you're objecting to is that you started relying on
> something *outside* the interface, and when you changed to using something
> that actually complied with it, you got hosed.

I switched from Pylons to Pyramid and got hosed. Pyramid is supposed
to be better than Pylons and more flexible, but in this case it's
worse and less flexible.

The Pyramid interface is less useful than Beaker's. That's a
limitation of the Pyramid interface, and can arguably be considered a
bug.

> In the meantime, AFAICT, it's reasonably simple to put a random id (or

> ... in the session

> dict. This would make the application run under any sessioning
> implementation.

That's what I ended up doing, but why not have a standard way to get
the session ID?

>> client-side sessions don't have an id, but server-side sessions require
>> it. because pyramid doesn't have an official interface to support this,
>> all server side session libraries need to implement this themselves --
>> somehow -- to support it. I stress "somehow". There's no way to ensure
>> or expect consistency in this across libraries. If you look at how
>> different session providers in the ecosystem have dealt with this, there
>> is little uniformity in approach. I've seen `session_id`,
>> `_session_id`, `sessionID` and a few other variations -- both as
>> attributes and internal dict values. If ISession had an official
>> `session_id` attribute, then people developing server side libraries
>> would adapt to that; there would be uniformity and portability. Those
>> using client-side sessions would never use or expect a session_id , and
>> not be inconvenienced.

I *tried* to write a 'pyramid_[session]' library...

I *tried* to modify 'pyramid_redis_sessions' to put the ID as
'session.id' or 'session["id"]''...

I *tried* to understand how Beaker did it...

But every time I got lost in how they generate the ID and manage the
cookie. And when I tried to just read the cookie in my tween, I
realized I would have to unsign it and get the secret... all the stuff
that the session library does, and why should I have to reimplement it
all? Why can't the library just have a METHOD that I can CALL?
Everything else in Pyramid does.

[Bert JW Regeer]

What are you using the session ID for?

Bert JW Regeer

[Chris McDonough]

On 04/28/2014 11:28 PM, Mike Orr wrote:
> On Mon, Apr 28, 2014 at 3:07 PM, Chris McDonough <chr...@plope.com> wrote:
>> On 04/28/2014 04:05 PM, Jonathan Vanasco wrote:
>>>
>> IMO, the point of an interface is to provide an abstract API which folks can
>> meet to provide alternate implementations. If server-side session
>> implementations had a session id, but client-side session implementations
>> did not, it would mean that *deployers* of an application could not switch
>> out the sessioning implementation from one to the other and still expect the
>> parts of their applications which rely on sessions to work, which is AFAICT
>> exactly what you're objecting to now.
>
> We don't need two interfaces. We just need an interface that says
> 'session.id' or 'session.get_session_id()' should return the session
> ID string, or None if it doesn't support session IDs or doesn't wish
> to expose it. Presto, everybody's problem is solved.

FWIW, I'd consider this problem created rather than solved in a scenario
that I'd like to support. I create applications that are themselves
frameworks (e.g. substanced). These applications rely on being able to
let deployers of those applications (note: these are not necessarily the
same people as the developers of all the various parts of the
application) configure in different sessioning implementations based on
their deployment requirements. For example, some systems are deployed
across multiple app servers, and the deployer may therefore need to use
cookie-based sessions or a shared backend session store to make sure the
same session data can be read by all of the appservers. These
applications also, in the meantime, allow pluggability of add-ons
written by unrelated people.

In this scenario, what happens when someone writes an add-on that (just
because it's available) uses the session id as a key into something, and
the deployer uses this add-on, and the session implementation the
deployer has configured in returns None from get_session_id? Nothing
good, AFAICT. What's the add-on code meant to do in this circumstance?
Either the add-on has to check for None as a return value from
get_session_id, or it has to check that the current sessioning
implementation implements the interface that supports a non-None return
value from get_session_id. And what if it does return None? What's the
add-in meant to do?

There is effectively no difference between this and having two
interfaces. In either case, the add-on implementer has to be aware that
not every sessioning implementation supports returning non-None session
ids. And you can bet that they won't, and instead of actually working,
the add-on is going to use None as a key into something that expects it
to be a unique session id value, and things are going to break in
strange ways. Detecting and fixing this breakage will be turned into
the deployer's problem when it really should be the application and/or
add-on developer's problem. Right now, without a get_session_id
interface, it just *is* the developer's problem, which puts the onus on
them to think about it, and this is the right division of responsibility
IMO.

FWIW, I've made similar mistakes in creating APIs in other pluggable
systems where I define a single interface that effectively requires a
capabilities check against its return value, and it has never ended
well. There are probably even some APIs like this in Pyramid; but it's
a poor pattern.

>> The only difference between this
>> scenario and the one you're objecting to is that you started relying on
>> something *outside* the interface, and when you changed to using something
>> that actually complied with it, you got hosed.
>
> I switched from Pylons to Pyramid and got hosed. Pyramid is supposed
> to be better than Pylons and more flexible, but in this case it's
> worse and less flexible.

You are confusing flexible with featureful, AFAICT. Not requiring a
sesion id is more flexible than requiring one. Smaller APIs are
generally more flexible than larger ones.

You seem to be disappointed that you had to add code to maintain a
session id. Given that you are *not* writing a pluggable system, and
you control the horizontal and vertical of both the application code and
the deployment, how much code was it? Was it more code than a function
like this?

def get_session_id(session):
return session.setdefault('myapp.sessionid', uuid.uuid4())

Replace uuid.uuid4() with whatever makes sense for your system, of
course. Note that this is more flexible than Pyramid mandating the
composition of a session id, because you can make it whatever the heck
you want, as long as it meets the uniqueness requirements for your
deployment.

> The Pyramid interface is less useful than Beaker's. That's a
> limitation of the Pyramid interface, and can arguably be considered a
> bug.
>
>> In the meantime, AFAICT, it's reasonably simple to put a random id (or
>> ... in the session
>> dict. This would make the application run under any sessioning
>> implementation.
>
> That's what I ended up doing, but why not have a standard way to get
> the session ID?

No one, AFAICT, has described what they actually use the session id for.
If you have, I'm sorry, and I've missed it. But all the traffic about
this seems a bit misplaced without some examples.

>>> client-side sessions don't have an id, but server-side sessions require
>>> it. because pyramid doesn't have an official interface to support this,
>>> all server side session libraries need to implement this themselves --
>>> somehow -- to support it. I stress "somehow". There's no way to ensure
>>> or expect consistency in this across libraries. If you look at how
>>> different session providers in the ecosystem have dealt with this, there
>>> is little uniformity in approach. I've seen `session_id`,
>>> `_session_id`, `sessionID` and a few other variations -- both as
>>> attributes and internal dict values. If ISession had an official
>>> `session_id` attribute, then people developing server side libraries
>>> would adapt to that; there would be uniformity and portability. Those
>>> using client-side sessions would never use or expect a session_id , and
>>> not be inconvenienced.
>
> I *tried* to write a 'pyramid_[session]' library...
>
> I *tried* to modify 'pyramid_redis_sessions' to put the ID as
> 'session.id' or 'session["id"]''...
>
> I *tried* to understand how Beaker did it...
>
> But every time I got lost in how they generate the ID and manage the
> cookie. And when I tried to just read the cookie in my tween, I
> realized I would have to unsign it and get the secret... all the stuff
> that the session library does, and why should I have to reimplement it
> all? Why can't the library just have a METHOD that I can CALL?
> Everything else in Pyramid does.

I hope the above and my prior emails explain the why-not.

- C

[wilk]

On 29-04-2014, Bert JW Regeer wrote:
>
> --Apple-Mail=_80215049-92A3-41FB-982F-7A0CE7A05959
> Content-Transfer-Encoding: quoted-printable
> Content-Type: text/plain;
> charset=windows-1252
>
> So this discussion has been going on for quite some time, and each time =
> I saw this email come by I figured there has to be a better way than to =
> continue going on whether there should be an session_id on the session =
> or not.
>
> I personally don=92t think there should be, and it should be =
> implementation defined, so to that end I have actually ended up writing =
> pyramid_pluggable_session. It=92s really simple, it even takes the same =
> parameters as the default SignedCookieSessionFactory(), it couldn=92t be =
> simpler =85 ;-)

>
> So here you can find the code:
>
> https://github.com/usingnamespace/pyramid_pluggable_session

You are the pyramid session champion !


--
William

[Jonathan Vanasco]

On Monday, April 28, 2014 11:46:22 PM UTC-4, Bert JW Regeer wrote:

What are you using the session ID for? 

Performance logging (session_id into statsd and logs).  Unit Testing. Integrated Testing. Automated Test suites. Development Troubleshooting ; Production Customer support.  

Often times we need the actual SessionID, which is used as the storage backed identifier, to access the stored value.  It's annoyingly complicated to working around this with custom classes and monkeypatches for each session-ing option. 

Beyond that, I can't make this clear enough... Personal opinions on Interfaces / Client vs Server / etc aside... I'd like to talk about common expectations :

1. Server Side Sessions are historically common across the internet.  Server Side Sessions are basically how shit got done for decades.  Developers expect them.  Jump on StackOverflow and you'll see thousands of questions across a variety of languages for people talking specifically about the session_id.

2. How other popular python projects deal with session ids: 

Beaker Session 

( used in Bottle, Turbogears, Pylons )

.id 

Cherrrypy

.id

Django:

old - request.session.session_key

new - request.session._session_key

Flask:  

.sid

plone

getBrowserId()

tornado

no standardized session support

multiple endorsed addons provide session support , all seem to provide id but differently

web2py

response.session_id

werkzeug

.sid

web.py

.session_id

webapp2

.sid

zope:

externally getClientId

internally client_id

Pyramid appears to be the ONE AND ONLY python web framework that supports sessions BUT does not support a session_id.  In fact, as stated above, the core developers oppose supporting a session id.

I don't mean to disagree or address any of the rationales/reasons mentioned above against supporting the session id.    Many of them are very good and sound.

However, if you want to know why this question comes up often , why it will continue to come up often and -- potentially -- why some people might be turned off by pyramid... it's the list above.  What is simple, normal, expected, trivial and standard across many frameworks is an "advanced topic" in Pyramid and left to developers to deal with on a per-project basis.    Pyramid is the least opinionated framework for just about everything -- but sessions.  

 

[Bert JW Regeer]

On Apr 29, 2014, at 14:30 , Jonathan Vanasco <jona...@findmeon.com> wrote:

> On Monday, April 28, 2014 11:46:22 PM UTC-4, Bert JW Regeer wrote:
> What are you using the session ID for?
>
> Performance logging (session_id into statsd and logs). Unit Testing. Integrated Testing. Automated Test suites. Development Troubleshooting ; Production Customer support.

I would say you are dual purposing the session ID. There is no guarantee that for example the session id is always unique within a certain timeframe. If one session ends, it is technically possible for a new session to be created with the same ID. Using it for logging, or any of the above seems like it would be setting yourself up for failure, or more log hunting than necessary.

If you need a unique identification, add it to the session yourself. request.session[‘unique_id’] = uuid.uuid4(). Do note that even Django has changed the session_key from session.session_key to session._session_key. Note that Python convention states:

_single_leading_underscore: weak "internal use" indicator.

Add the unique id, and problem is solved. This would solve it across the board, even on those session implementations that would return None (as suggested previously in this thread).

>
> Often times we need the actual SessionID, which is used as the storage backed identifier, to access the stored value. It's annoyingly complicated to working around this with custom classes and monkeypatches for each session-ing option.

Certain session implementations there is no storage backing it, such as the cookie only, in that case having a session id is completely useless, since it isn’t used. There is no way to retrieve the session information because it isn’t stored server side.

>
> Beyond that, I can't make this clear enough... Personal opinions on Interfaces / Client vs Server / etc aside... I'd like to talk about common expectations :
>
> 1. Server Side Sessions are historically common across the internet. Server Side Sessions are basically how shit got done for decades. Developers expect them. Jump on StackOverflow and you'll see thousands of questions across a variety of languages for people talking specifically about the session_id.

Yes, and a lot of those answers actually are:

“Sorry, that isn’t possible, add in your own unique identifier”.

See: http://stackoverflow.com/a/15187430/13986 (Flask session id)

>
> 2. How other popular python projects deal with session ids:
>
> Beaker Session
> ( used in Bottle, Turbogears, Pylons )
> .id
>
> Cherrrypy
> .id
>
> Django:
> old - request.session.session_key
> new - request.session._session_key
>
> Flask:
> .sid

Flask, also uses signed cookies, see: https://github.com/mitsuhiko/flask/blob/master/flask/sessions.py there is no .sid variable available to get access to the session ID. The only .sid I can find is in one particular implementation of a file backed session manager, which is an implementation detail. It is not part of the official Flask session support.

>
> plone
> getBrowserId()
>
> tornado
> no standardized session support
> multiple endorsed addons provide session support , all seem to provide id but differently
>

Tornado as you stated doesn’t have any session libraries out of the box, and thus it is strictly implementation defined. When using secure cookies with tornado however no session ID is provided.

> web2py
> response.session_id

If you are using the cookie based session storage, response.session_id will be True for every single last session:

https://github.com/web2py/web2py/blob/7592cd2fe031509446042d5c768c90116ebfaf9c/gluon/globals.py#L803

In other words, not exactly something to rely on, and definitely not something used for logging. Once again, session id existing and being useful is implementation defined.

>
> werkzeug
> .sid

Werkzeug has no default session implementation. There are two implementations that can equally be used for storing session information. The contrib/sessions.py or the contrib/securecookies.py. The latter does not provide a session ID.

>
> web.py
> .session_id
>
> webapp2
> .sid
>

This depends on the backend being used. If you use the default secure cookie storage, there is no session ID available.

https://code.google.com/p/webapp-improved/source/browse/webapp2_extras/sessions.py#177

> zope:
> externally getClientId
> internally client_id
>

I didn’t care to go digging into the Zope source tree...

>
> Pyramid appears to be the ONE AND ONLY python web framework that supports sessions BUT does not support a session_id. In fact, as stated above, the core developers oppose supporting a session id.

Pyramid is not the only one that doesn’t support session ID’s, as proven above. Many of the above implementations use the secure cookie method that Pyramid is also using, and with the exception of Beaker (they are generated and then stored as data within the session itself), do not provide a session ID when using the secure cookie method.

Even with the server side storage ones, depending on the implementation the session ID may or may not be available, just like in Pyramid it is implementation defined based upon the current session provider being used. Relying on the session ID being available (especially in web2py…) is simply not a good idea. It is a much better idea to store a unique value into the session yourself.

>
> I don't mean to disagree or address any of the rationales/reasons mentioned above against supporting the session id. Many of them are very good and sound.
>
> However, if you want to know why this question comes up often , why it will continue to come up often and -- potentially -- why some people might be turned off by pyramid... it's the list above. What is simple, normal, expected, trivial and standard across many frameworks is an "advanced topic" in Pyramid and left to developers to deal with on a per-project basis. Pyramid is the least opinionated framework for just about everything -- but sessions.

Except as I have just proven that this is’t normal, expected or trivial and standard across many frameworks. The session ID existing isn’t a guarantee by a long shot. Add a NewRequest handler that fires, have it check the session for your unique ID, and if it doesn’t exist, add it to the session. Or add a new reified method to request that checks the session for this unique ID, and if it doesn’t exist create one and set it in the session, and return it. Then request.unique_id will return that sessions unique ID, that can be used for logging.

This isn’t an advanced topic in Pyramid, and I would argue it isn’t even a limitation. There are many solutions all of them equally simple to implement. My pyramid_pluggable_session allows you to very simple plug in new backends, and has a _session_id available on it, start using it across all your applications and you will have access to a session id that is unique. It’s unlikely there will be a collision in 20 bytes of random anytime soon.

With sessions Pyramid is also not opinionated. Out of the box it provides a very sane default, and users are encouraged to find other session implementations that match their needs. The ISession interface contains everything required, application developers can layer any other functionality required on top to customise sessions to their needs/requirements for their application. Build a vanasco_sessions module, upload it to pypi and depend on it in all your future Pyramid applications. Now you can always be guaranteed that a session ID variable exists in your session.

Bert JW Regeer

[Jonathan Vanasco]

The focus of this discussion shouldn't be the merits, drawbacks, or implementation details of server-side sessions vs client-side sessions -- which everyone is quick to chime in on.  We can talk about that all day.  We have, many times..

My concern isn't that there is a session_id variable in every session or the uniqueness of said id.

My concern is that there should be standard way in which Pyramid plugins use and expose the raw session_id of server side sessions.  Because ISession doesn't have a session_id , everyone has implemented this differently.   Without some sort of official stance, everyone in the future will implement this differently.  A consensus on this would allow for server-side session libraries to be swappable -- switching from Redis to Memcached; or "pyramid_redis_a" to "pyramd_redis_b".

I understand that "client side sessions" are "basic enough".  However, server-side sessions all rely on a raw cookie value as their session id.   It's a technological given.  My concern isn't on the merits of whether or not that ID should be trusted, if it is valuable, or what it should be used for.   My concern is the lack of a standard in the pyramid ecosystem for dealing with this common situation.  

Instead of having a familiar api when dealing with sessions or writing new packages, and more interoperability / swap-ability of plugins... the project is opting for a "no rules" approach on this matter, stating that the client side sessions are basic enough.  That doesn't sound like a compelling argument to me.   

And instead of addressing the fact that server-side plugins are left to approach this differently -- and have been doing just that -- the utility and motives of the data are being questioned.  Even if the underlying session_id were 100% worthless, there would still be tangible benefits if every session library in the pyramid ecosystem tracked that variable in an unified way ( readability, familiarity, maintenance and collaboration between projects ).

Unlike many other Python frameworks which left the entirety of sessions to third parties, or left their sessions entirely undocumented and largely unreliable, Pyramid crafted a richly documented sessioning system with an official interface/api for implementation and an expectation for libraries to be built upon it.  The interface is exacting on how to deliver on virtually every core sessinoning feature.  The sole omission -- how to provide for the session_id in server-side sessions.  

That being said, if you want to talk about the utility of a session_id variable:

We actually prefer user and "logged-in-event" logging , and do that with our own Business Intelligence systems.  The latter is essentially what Bert described ( though we use a serial instead of a uuid ).

The desired "session_id" logging gives parity from our Python logs to our webserver logging -- which can only access raw header/cookie values.  Third party analytics systems, driven by PHP's popularity, expect to see this and will segment on the logged session-id.  

Some of the proposed solutions wouldn't solve our needs, because they stash a unique ID within the session.  That solves a completely different problem.  Placing a UID/UUID within the session data works perfectly for accessing and utilizing a UUID for a session within python.  But one must use a specific session library to load and extract that identifier from the session payload.  That's not what I'm interested in.  This internal UID is not available on the proxy/gateway server nor is it available on nginx/apache, mod_security, other server plugins or various other server systems that touch our http traffic.  Those can only access the raw plaintext cookie value.  The internal uuid is not necessarily available to python outside of Pyramid either, unless one goes so far as to create a middleware package to offer it onto other sections of the WSGI stack.   I am interested in the raw cookie value, which appears across all the various systems in our technology stack and is used as the key in the datastore ( or can be computed to derive that key ).  

Also in the "utility" column -- when dealing with "realtime" performance logging systems like Cube / StatsD, we use the SessionID to give us transparency across every element in our stack. Everything can access it, so everything can log based on it.  We also insert a uuid into the header on the gateway to serve as a "request_id".  session data is often more useful than request data though.  Usually you don't want to use "highly unique" values for StatsD, but we discovered a handful of use-cases for using the SessionID to identify bad behavior, or track session activity within a subset of URL paths.

To close out a few sidenotes:

- I thought i was clear that I was talking specifically about Server Side sessions, not client side ones.  

- I am fully aware that Client Side sessions do not have a ID.  I'm not advocating that they offer an id or should offer an id.  I am advocating for a 

unified session_id interface for server side sessions.  

- When it comes to server side sessions, virtually every framework has a documented "session_id" variable OR a general consensus on what to use when developing Plugins.  

- Flasl does use signed cookies by default.  BUT there is an extensive section on the docs for Server Side Session support.  All the server side sessions examples in the Flask documentation use .sid ( http://flask.pocoo.org/snippets/category/sessions/ )

- I based my tornado comment by going through the sourcecode of multiple session libraries from the Tornado wiki.

- Bert's session package is great, but also illustrates this issue.  He made an arbitrary decision to use `_session_id`.  Packages released under his system would be largely compatibile with one another; but since there is no official implementation from pyramid, the session_id  should be expected to NOT be compatible with other pyramid session packages.  

[Michael Merickel]

Ignoring everything else for the moment, you've already made yet another assumption that you're not going to be happy about. The raw cookie value is in no way a session identifier. For example in pyramid_redis_sessions the session identifier is cryptographically signed to prevent clients from even attempting to guess other session identifiers, and arguably it should always be signed. And pyramid_redis_sessions also pickles the contents so its unusable outside of Python by default (while providing hooks to change that if you need).

The Pyramid API is *more* flexible than what you want, despite claiming that it is less flexible. It is all a matter of perspective but the API itself is less constraining than you want. It's clear from your requirements that you should be implementing your own internal extensions to the ISession because Pyramid has no intention of dictating things like cookie values for usage outside of Pyramid. That is the exact reason that these hooks exist.

[Bert JW Regeer]

On Apr 30, 2014, at 13:29 , Jonathan Vanasco <jona...@findmeon.com> wrote:

> […]
>

This seems to be going in circles. session id is not common across all implementations, doesn’t need to exist for most of them, and thus shouldn’t be relied on. Instead stick your own unique token into the session if you want it.

You haven’t convinced me that session_id should be added to the ISession interface.

> The desired "session_id" logging gives parity from our Python logs to our webserver logging -- which can only access raw header/cookie values. Third party analytics systems, driven by PHP's popularity, expect to see this and will segment on the logged session-id.

Even with my session library, which offers a session ID, you can’t just pull the session ID out of the cookie since it is JSON serialized and signed. The cookie is of no use what so ever for tracking. Even on PHP systems if one for example has PHP SuHoSin installed the session cookie is automatically securely signed/encrypted. Once again, the web server has no access to the session id.

If you want to track a request from front-end to backend add another HTTP header that gets logged on the front-end/back-end. Just like Cloudflare does with their CF Ray ID header. Then you can track web server hits to backend hits without relying on an implementation specific cookie value! Note once again that the cookie contents for a particular session are implementation defined, there is no standard across them at all!

>
> […]

>
> - Bert's session package is great, but also illustrates this issue. He made an arbitrary decision to use `_session_id`. Packages released under his system would be largely compatibile with one another; but since there is no official implementation from pyramid, the session_id should be expected to NOT be compatible with other pyramid session packages.

That is why it has an underscore prepended, this makes it an internal value that can not be relied on, unless you always use my session package. The nice thing is that with mine you can keep using it, as long as you swap out the plugs, and they won’t ever alter the fact that there is a _session_id.

Unless your session cookie (sent to the browser) is just the session ID, without signing (which allows users to trivially change the session ID associated with their connection, which is bad juju if you don’t want them trying to gain access to an administrative session for example by iterating over session id’s), the session cookie will never contain just the session id. Thus using that to track requests across web server to back-end doesn’t make sense either.

Bert

[Chris McDonough]

On 04/30/2014 03:29 PM, Jonathan Vanasco wrote:
> The focus of this discussion shouldn't be the merits, drawbacks, or
> implementation details of server-side sessions vs client-side sessions
> -- which everyone is quick to chime in on. We can talk about that all
> day. We have, many times..
>
> My concern isn't that there is a session_id variable in every session or
> the uniqueness of said id.
>
> My concern is that there should be standard way in which Pyramid plugins
> use and expose the raw session_id of server side sessions. Because
> ISession doesn't have a session_id , everyone has implemented this
> differently. Without some sort of official stance, everyone in the
> future will implement this differently. A consensus on this would allow
> for server-side session libraries to be swappable -- switching from
> Redis to Memcached; or "pyramid_redis_a" to "pyramd_redis_b".
>
> I understand that "client side sessions" are "basic enough". However,
> server-side sessions all rely on a raw cookie value as their session id.
> It's a technological given. My concern isn't on the merits of
> whether or not that ID should be trusted, if it is valuable, or what it
> should be used for. My concern is the lack of a standard in the
> pyramid ecosystem for dealing with this common situation.

By the way, thank you for keeping up with this discussion, I know
debating about things with folks who don't share your point of view can
get pretty boring and frustrating.

As a thought experiment, let's imagine that Pyramid cookie based
sessions did have a session id (probably just a random one), and we
changed the ISession interface to require that all sessioning
implementations exposed the session id as session.id, and that we
mandated that a session id must be a string.

> Some of the proposed solutions wouldn't solve our needs, because they
> stash a unique ID within the session. That solves a completely
> different problem. Placing a UID/UUID within the session data works
> perfectly for accessing and utilizing a UUID for a session within
> python. But one must use a specific session library to load and extract
> that identifier from the session payload. That's not what I'm
> interested in. This internal UID is not available on the proxy/gateway
> server nor is it available on nginx/apache, mod_security, other server
> plugins or various other server systems that touch our http traffic.
> Those can only access the raw plaintext cookie value. The internal
> uuid is not necessarily available to python outside of Pyramid either,
> unless one goes so far as to create a middleware package to offer it
> onto other sections of the WSGI stack. I am interested in the raw
> cookie value, which appears across all the various systems in our
> technology stack and is used as the key in the datastore ( or can be
> computed to derive that key ).

The presumption that there is a session.id variable won't solve this
particular problem, as Bert and Mike have already said. But as a
separate problem, it could be "solved" if we mandated that a session id
be made available in a common way in the cookie that represents the
session token across all implementations. This is a big ask, though, if
only because it means that folks who don't actually want to expose the
raw session id could not create an interface-compliant implementation.
In my experience, it also generally makes systems less flexible when you
begin to tell potential implementers how they must do things rather than
only telling them what they must do.

> Also in the "utility" column -- when dealing with "realtime" performance
> logging systems like Cube / StatsD, we use the SessionID to give us
> transparency across every element in our stack. Everything can access
> it, so everything can log based on it. We also insert a uuid into the
> header on the gateway to serve as a "request_id". session data is often
> more useful than request data though. Usually you don't want to use
> "highly unique" values for StatsD, but we discovered a handful of
> use-cases for using the SessionID to identify bad behavior, or track
> session activity within a subset of URL paths.

Would the addition of session.id help here or does your stack have the
same problem where the session id must be discoverable outside Pyramid?

>
>
> To close out a few sidenotes:
>
> - I thought i was clear that I was talking specifically about Server
> Side sessions, not client side ones.
>
> - I am fully aware that Client Side sessions do not have a ID. I'm not
> advocating that they offer an id or should offer an id. I am advocating
> for a
> unified session_id interface for server side sessions.
>
> - When it comes to server side sessions, virtually every framework has a
> documented "session_id" variable OR a general consensus on what to use
> when developing Plugins.
>
> - Flasl does use signed cookies by default. BUT there is an extensive
> section on the docs for Server Side Session support. All the server
> side sessions examples in the Flask documentation use .sid (
> http://flask.pocoo.org/snippets/category/sessions/ )
>
> - I based my tornado comment by going through the sourcecode of multiple
> session libraries from the Tornado wiki.
>
> - Bert's session package is great, but also illustrates this issue. He
> made an arbitrary decision to use `_session_id`. Packages released
> under his system would be largely compatibile with one another; but
> since there is no official implementation from pyramid, the session_id
> should be expected to NOT be compatible with other pyramid session
> packages.
>

> --
> You received this message because you are subscribed to the Google
> Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pylons-discus...@googlegroups.com

> <mailto:pylons-discus...@googlegroups.com>.

> To post to this group, send email to pylons-...@googlegroups.com

> <mailto:pylons-...@googlegroups.com>.

[Michael Merickel]

On Wed, Apr 30, 2014 at 3:28 PM, Chris McDonough <chr...@plope.com> wrote:

The presumption that there is a session.id variable won't solve this particular problem, as Bert and Mike have already said.  But as a separate problem, it could be "solved" if we mandated that a session id be made available in a common way in the cookie that represents the session token across all implementations.  This is a big ask, though, if only because it means that folks who don't actually want to expose the raw session id could not create an interface-compliant implementation. In my experience, it also generally makes systems less flexible when you begin to tell potential implementers how they must do things rather than only telling them what they must do.

We also shouldn't forget that sessions do not require a cookie, even if that's the primary way of tracking them across requests. For Pyramid to dictate how a session must be tracked would be a huge loss in the flexibility of the API. Currently the factory has access to the entire request to decide where the session id is at, allowing implementers to do what they need to do to find the tracker from a cookie or a query string or some higher-level mechanism.

[Chris McDonough]

On 04/30/2014 04:36 PM, Michael Merickel wrote:
> On Wed, Apr 30, 2014 at 3:28 PM, Chris McDonough <chr...@plope.com

> <mailto:chr...@plope.com>> wrote:
>
> The presumption that there is a session.id <http://session.id>

Good point.

- C

[wilk]

On 30-04-2014, Bert JW Regeer wrote:

About _session_id of pyramid_pluggable_session

> That is why it has an underscore prepended, this makes it an internal

> =
> value that can not be relied on, unless you always use my session =
> package. The nice thing is that with mine you can keep using it, as long =
> as you swap out the plugs, and they won=92t ever alter the fact that =
> there is a _session_id.

Sorry it's not clear for me, when i write a new plug can i rely on
_session_id ?

--
William

[Bert JW Regeer]

Yes. You can rely on _session_id existing. From an outside perspective, anyone using request.session SHOULD NOT rely on _session_id existing, since it may not be pyramid_pluggable_session, and thus _session_id may not be available.

Bert

[wilk]

On 01-05-2014, Bert JW Regeer wrote:
>
> --Apple-Mail=_8BFE3C8B-DFF0-491C-A72C-E7137EA10853
> Content-Transfer-Encoding: quoted-printable
> Content-Type: text/plain;
> charset=iso-8859-1

>
>
> On May 1, 2014, at 12:39 , wilk <wi...@flibuste.net> wrote:
>
>> On 30-04-2014, Bert JW Regeer wrote:

>>=20
>> About _session_id of pyramid_pluggable_session
>>=20
>>> That is why it has an underscore prepended, this makes it an internal=20=
>
>>> =3D
>>> value that can not be relied on, unless you always use my session =3D
>>> package. The nice thing is that with mine you can keep using it, as =
> long =3D
>>> as you swap out the plugs, and they won=3D92t ever alter the fact =
> that =3D
>>> there is a _session_id.
>>=20
>> Sorry it's not clear for me, when i write a new plug can i rely on=20
>> _session_id ?
>>=20
>> --=20
>> William
>>=20
>
> Yes. You can rely on _session_id existing. =46rom an outside =
> perspective, anyone using request.session SHOULD NOT rely on _session_id =
> existing, since it may not be pyramid_pluggable_session, and thus =

> _session_id may not be available.

I understand, thanks

--
William

[Jonathan Vanasco]

On Wednesday, April 30, 2014 4:28:49 PM UTC-4, Chris McDonough wrote:

By the way, thank you for keeping up with this discussion, I know
debating about things with folks who don't share your point of view can
get pretty boring and frustrating.

Thanks, and right back at you on both points.  I know and admit that I am the taxing and annoying person here!

My rationale for all this argument isn't for my benefit.  whenever i use pyramid I can (and do) write in my needed functionality.  i can patch/fork/subclass in most needed functionality.  i just worry because this is still a common situation and common need.  with a bit of leadership -- even if only in documentation  -- the pyramid ecosystem would get a lot more interoperability and avoid problems with newer devs.

Next to that... 

Correct on all things that you and Mike noted :

1. Sessions might not have a cookie - they could be implemented with a URL fragment.  That was popular years ago, when browsers might not support cookies and developers didn't worry as much about the effect of URL elements on search engines or browser security (the url is often logged , headers are not).

2.Some sessions schemes do have 2 session identifiers :

    a) a public session_id , transmitted in plaintext and typically used in cookies

    b) a private session_id , a value computed from the public session_id, which is mapped to a backend storage. 

The standard beaker cookie is actually a concatenation of a digest + the session id for some lightweight validation.  The underlying session_id is in plaintext within the cookie, but it's prefixed by some other data.

There are a handful of dual-key implementations that encode the underlying session id.   most often the public session_id is just the backend identifier that has either been encrypted or digitally signed (with an expiry time, ip address, etc).  these patterns are less like the classic Server Side Session pattern and more like a Client Side Session pattern with a backend payload identifier.  In the case above the payload is a server side session id; but often times it's an internal "user id" or other data used to sync logged-in activities across devices by the same user.  

now let me respond to these 2 notes you made :

 

The presumption that there is a session.id variable won't solve this
particular problem, as Bert and Mike have already said.  But as a
separate problem, it could be "solved" if we mandated that a session id
be made available in a common way in the cookie that represents the
session token across all implementations.  This is a big ask, though, if
only because it means that folks who don't actually want to expose the
raw session id could not create an interface-compliant implementation.
In my experience, it also generally makes systems less flexible when you
begin to tell potential implementers how they must do things rather than
only telling them what they must do.

..... 

Would the addition of session.id help here or does your stack have the
same problem where the session id must be discoverable outside Pyramid?

Let's assume that I have a plaintext session id in the cookie , named "SessionID".  

When I'm looking throughout the stack for the current SessionID, I can just look at the header value which is associated with the incoming GET/POST request.  Great.

When I'm in Pyramid I can look at the incoming cookies but... here's the annoying part... that cookie may or may not belong to the current session.

The SessionID on the incoming session may have been invalidated by an underlying timeout, ip mismatch, backend cache miss; or even a developer calling 'invalidate'.    The cookie value for the new, current session, would be in the response object.  

because of this behavior (which is good), one needs to search in multiple places in order to find the cookie associated with the current session.

here's a sample gist...

https://gist.github.com/jvanasco/aa6a844ee5f27d37f588

after going through this dialog and example, i think i can better state the frustration / what I think the shortcoming is, and in a way that applies to client and server based sessions alike...

The issue is not really about deriving the underlying session_id, which may or may not be encoded.  Though it would be great if that existed.  This is perhaps a side-topic.  

The issue is in not having a direct association between a "session object" and the "cookie value" which created it. 

A client-side session would always correspond to the incoming cookie value.  

A server-side session could be associated with a cookie value that appears in either the request or response cookies -- or neither in the case of multiple invalidations.

There's simply nothing that ties the originating cookie value to a given session object. 

[Michael Merickel]

On Fri, May 2, 2014 at 2:42 PM, Jonathan Vanasco <jona...@findmeon.com> wrote:

The issue is in not having a direct association between a "session object" and the "cookie value" which created it. 

A client-side session would always correspond to the incoming cookie value.  

A server-side session could be associated with a cookie value that appears in either the request or response cookies -- or neither in the case of multiple invalidations.

There's simply nothing that ties the originating cookie value to a given session object. 

It seems the struggle here is that the session is not exactly what you'd like to be tracking in most cases, but rather individual activity over time. Since sessions have a defined lifetime, security issues, etc they do not pass around a raw value and even if they do it's possible and likely that the value is not valid any longer. For example, a client can send an expired session cookie back as long as it damn well pleases and it's up to the server to ignore it. For tracking this just isn't great. At the end of the day it really seems to me like you need another cookie/identifier to accomplish your goals.

[Chris McDonough]

On 05/02/2014 03:42 PM, Jonathan Vanasco wrote:
> My rationale for all this argument isn't for my benefit. whenever i use
> pyramid I can (and do) write in my needed functionality. i can
> patch/fork/subclass in most needed functionality. i just worry because
> this is still a common situation and common need. with a bit of
> leadership -- even if only in documentation -- the pyramid ecosystem
> would get a lot more interoperability and avoid problems with newer devs.

Sure. It's always a tradeoff, though. If we over-specify the session
interface, it makes session implementers' lives harder, and may reduce
flexibility for consumers.

> Next to that...
>
> Correct on all things that you and Mike noted :
>
> 1. Sessions might not have a cookie - they could be implemented with a
> URL fragment. That was popular years ago, when browsers might not
> support cookies and developers didn't worry as much about the effect of
> URL elements on search engines or browser security (the url is often
> logged , headers are not).

Not a fragment (these are never sent to the server), but yeah, a URL
segment or query string.

> 2.Some sessions schemes do have 2 session identifiers :
> a) a public session_id , transmitted in plaintext and typically
> used in cookies
> b) a private session_id , a value computed from the public
> session_id, which is mapped to a backend storage.
>
> The standard beaker cookie is actually a concatenation of a digest + the
> session id for some lightweight validation. The underlying session_id
> is in plaintext within the cookie, but it's prefixed by some other data.
>
> There are a handful of dual-key implementations that encode the
> underlying session id. most often the public session_id is just the
> backend identifier that has either been encrypted or digitally signed
> (with an expiry time, ip address, etc). these patterns are less like
> the classic Server Side Session pattern and more like a Client Side
> Session pattern with a backend payload identifier. In the case above
> the payload is a server side session id; but often times it's an
> internal "user id" or other data used to sync logged-in activities
> across devices by the same user.

Yes, lots of strategies to ensure that the session id being sent is valid.

> now let me respond to these 2 notes you made :
>

> The presumption that there is a session.id <http://session.id>

> variable won't solve this
> particular problem, as Bert and Mike have already said. But as a
> separate problem, it could be "solved" if we mandated that a session id
> be made available in a common way in the cookie that represents the
> session token across all implementations. This is a big ask,
> though, if
> only because it means that folks who don't actually want to expose the
> raw session id could not create an interface-compliant implementation.
> In my experience, it also generally makes systems less flexible when
> you
> begin to tell potential implementers how they must do things rather
> than
> only telling them what they must do.
> .....
>

> Would the addition of session.id <http://session.id> help here or

> does your stack have the
> same problem where the session id must be discoverable outside Pyramid?
>
>
> Let's assume that I have a plaintext session id in the cookie , named
> "SessionID".

I'd rather assume you didn't, at least for purposes of answering the
second question above. As I'm sure you know, being able to parse the
session id out of a raw cookie value is definitely distinct from being
able to access the session's id as "session.id", and in the spirit of
the history of this thread, and the passion put forth in arguments for
the it, I'm at this point really just trying to figure out whether
there's benefit to mandating that ISession implementers must supply an
ID available as "session.id". In the interest of being clear, we should
settle that first.

Let's similarly put this third topic on the backburner while we figure
out whether it would benefit you to have a "session.id" attribute common
to all session implementations.

- C

[Mike Orr]

In my case it's fine because it just stores the None in the access
record. That loses the individual sessions, but at least some of the
others have session IDs so I can measure their behavior. That's better
than having no sessions at all, in which case I can't measure
anything.

There may be cases that break if there's ever a None, but (1) it's the
sysadmin's responsibility to choose a backend that has session IDs,
and (2) we haven't found any use cases for this, whereas my logging
and Jonathan's testing are actual use cases.

So all we're talking about is defining a place for the session ID if
it exists, not saying it's required. There could even be a
'has_session_ids' constant if you want to know whether the current
backend supports it. That could allow you to raise an exception on
startup if it's using the wrong backend.

> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to pylons-discus...@googlegroups.com.
> To post to this group, send email to pylons-...@googlegroups.com.

> Visit this group at http://groups.google.com/group/pylons-discuss.
> For more options, visit https://groups.google.com/d/optout.

--
Mike Orr <slugg...@gmail.com>

[Chris McDonough]

You haven't found a need to support any use cases similar to the one I
describe above where there's at least three separate individuals
involved (the application developer, the add-on developer, and the
deployer) who all must be able to rely on a shared understanding of the
ISession spec for the system to work ok when they combine their work.
That's fair enough, and I see your point, which is, I believe:

It's maximally convenient for me to be able to rely on a session.id
attribute: I need it for logging, and it's a hassle to need to define
this get_session_id function and use it instead of session.id:

def get_session_id(session):
return session.setdefault('myapp.sid', uuid4())

Also, it's a hassle for new Pyramid users to know that they'll
have to define such a function if they want to be able to rely
on a per-session key. They're used to systems which supply
an id for them already.

I think, however, that you haven't found any need to support this use
case because you in your environment you always wear all three hats (app
developer/add-on developer/deployer).

If I change the ISession spec to require an ID, or to require that some
API return a valid session id or None, it will probably make your life
easier, and probably will make easier the life of the new Pyramid user
in a similar development situation.

However, it will make the life of each person who only wears one of the
hats I describe harder. It will make the ISession implementer's life
harder because he will have to supply a new API even if his
implementation does not need IDs, and older versions of his ISession
code will not be in compliance anymore. It will make the add-on
developer's life harder because she'll have to be aware that session.id
might be None, and when it is, the code may need to branch out to
something similar to the function I showed above, or she might decide to
just use that function always. It will make the deployer's life harder
because he may wind up trying to combine an add-on and an ISession
implementation that don't really work together due to confusion on the
part of one or the other.

In the meantime, since you control the horizontal and vertical, and
you're most often wearing all three hats, it's pretty easy for you to
write the above function and use it. I think you're doing that now
without any ill effect. You've already borne the cost of the pain, and
now you're arguing with the umpire on the behalf of new users, more or
less, right?

We could mandate session.id. But instead of adding session.id, we could
alternately make the situation for the new Pyramid user better by adding
something to the docs about the ISession API not having an id, describe
the "why", and show the snippet function above to let them know how to
get around it.

So I guess my argument for not necessarily requiring session.id is that
there are other ways to make it better for the new Pyramid user that
don't impose the burden described above on ISession/add-on developers
and deployers by placing a small docs-reading burden on the person who
might wear all three hats.

Does that make sense? Note that I really am trying to get to the heart
of this issue, not trying to be obstinate, just trying to make my point
understood.

- C

[Jonathan Vanasco]

On Saturday, May 3, 2014 7:02:46 AM UTC-4, Chris McDonough wrote:

Sure.  It's always a tradeoff, though.  If we over-specify the session
interface, it makes session implementers' lives harder, and may reduce
flexibility for consumers.

I agree there is a tradeoff, but I don't see how adding a specification of the sort that "If there is a session_id, it should be provided by this method : " is something that necessarily makes lives harder or reduces flexibility.  I think it could be the opposite .  Different frameworks, packages and plugins handle the concept of a session_id for server-side-sessions in vastly different means. With a dedicated API method, if a plugin writer wants to expose a 'session_id' , they won't have to search and find a name for the method.  They'll have a method name and a snippet of documentation to work with.  Keep in mind that server-side-sessions are still very popular, and they all require some sort of "session_id" in the backend.

It's entirely possible that creating a session_id could be a complex process that requires collision checking against a datastore and/or other methods; looking at how most of the popular sessioning packages are set up though, they all see to follow a loose pattern where the session_id is loaded/generated on instantiation of the session object.  

I'm also not primarily concerned with requiring package maintainers to integrate session_ids; just recommend that they do, and if they do... there should be a consistent API.   I think within a short amount of time, developers who wanted/needed to access the session_ids would build in the needed functionality and issue pull requests.  if a developer doesn't provide an id, then that package may be forked or replaced by a similar backend that does.

I'll table talking about that "other topic" directly, but want to bring up this indirect point that hits both :

Throughout the bulk of pyramid, there are clearly defined APIs to integrate other packages against.  one can get to the underlying data / methods / etc in clear and consistent means.  this is a significantly different approach than projects like Rails or Django, which isolate the 'consumer' from the underpinnings and rely on "magic" to expose on a selection of data.   it's for this reason that many people prefer pyramid.

If you don't mind, for clarity, I want to use the term "session_key" instead of "session_id" and "UUID" for the "internal id" for a bit...  I don't want to create confusion between the UUID, session_key and python's id().... which are all similarly named.

I appreciate the docs suggestion about generating a "UUID" within the session, which can be used for logging and other means.  I think that's a great way to handle "session ids" in the sense of a session being an "application user".  I'm still concerned ( and I think mike is too ) with being able to access the underlying session object's "key".   there are a large common set of use-cases and utilities that a sessions's "UUIID" and "key" can both fulfill. there are numerous advantages that your approach has over using a raw "key" as well.  I don't mean to discount any of them.  I should, actually, rewrite some parts of our code to use them instead.

however, an internal "UUID" is still an alternative to having a consistent API that offers access to the key of the session object.  that "key" is still something that all server-side-sessions require by design; and that all packages provide differently.  regardless of the reasons that people need to use it or the alternative options that could provide much of the same functionality, without an API endpoint that developers could correlate this data to... this "session_key" becomes a "magic" element and a bit of an advanced topic to people who develop on Pyramid.

[Chris McDonough]

This conversation seems to be going in circles a bit.

Rather than debating the idea in the abstract, I'm trying to figure out
how you, in particular, would actually concretely make use of
"session.id" (or session.key or whatever).

This question needs to be considered independently from:

- being able to parse the key out of a cookie value "by hand". This
is unrelated to having an API on the session object itself.
Being able to do this is another topic which we need to
consider independently.

- being able to use the key as a lookup value into a sessioning
implementation. There is currently no ISession API to pass the key
to obtain information about the session, and adding such an
API is yet another topic.

Can anyone give me a concrete usage that's not solved by using a UUID
key stored in the session data itself that does not involve the 2
constraints above?

- C

[Randall Leeds]

On May 5, 2014 6:00 PM, "Chris McDonough" <chr...@plope.com> wrote:
>
>
> This conversation seems to be going in circles a bit.
>
> Rather than debating the idea in the abstract, I'm trying to figure out how you, in particular, would actually concretely make use of "session.id" (or session.key or whatever).
>
> This question needs to be considered independently from:
>
>   - being able to parse the key out of a cookie value "by hand".  This
>     is unrelated to having an API on the session object itself.
>     Being able to do this is another topic which we need to
>     consider independently.
>
>   - being able to use the key as a lookup value into a sessioning
>     implementation.  There is currently no ISession API to pass the key
>     to obtain information about the session, and adding such an
>     API is yet another topic.
>
> Can anyone give me a concrete usage that's not solved by using a UUID key stored in the session data itself that does not involve the 2 constraints above?

I've been trying and the issue is that any such scenario is just about configuration.

For instance, if I were going to build some analytics module that shipped data off to be indexed I might include a session id. If I were to publish this as a package today I might use a setting to configure the key of this value in the session data. I would probably use a setdefault call with a uuid in a NewRequest subscriber.

But as soon as we have more than one such need, I'd be configuring each one and each one must implement this configuration code.

So the issue comes to when there is >1 included pyramid plugin that wants to uniquely identify the session without adding more items that strictly necessary.

My resistance to jumping in completely on the side of adding API has been that I don't like making anyone deal with a maybe-None return value and I don't like causing extra items to be stored in the session (cookie storage, in particular, should be kept small).

But this seems quite parallel to csrf token.

I would ask, if you were to redesign it today, would your ideal ISession interface have csrf methods or would you want all the code that uses it to check for _csrf_ key in the session and create a random one if it doesn't exist?

I imagine things like the check_csrf_token predicate would have an additional keyword for key were it not for an ISession API which makes it opaque.

[Chris McDonough]

On 05/05/2014 10:36 PM, Randall Leeds wrote:
> On May 5, 2014 6:00 PM, "Chris McDonough" <chr...@plope.com

> <mailto:chr...@plope.com>> wrote:
> >
> >
> > This conversation seems to be going in circles a bit.
> >
> > Rather than debating the idea in the abstract, I'm trying to figure
> out how you, in particular, would actually concretely make use of

> "session.id <http://session.id>" (or session.key or whatever).

You're right. If I redesigned it, I might consider requiring that
ISession expose an get_uuid API which, under the hood, returned a UUID
associated with the session. It would mean satisfying your suggested
requirement that there be only one preferred way to do it for add-on code.

I would particularly use a time machine to so if I thought it would
satisfy Mike's and Jonathan's requirements. I don't think it does
though, because this API would not necessarily return the "internal"
session id, which is the key that the sessioning implementation actually
uses to look up the data. We keep hearing about needing access to the
"internal" session id from Jonathan at least, so there's more going on
there, and this seems to not be helpful for him, which is why I keep
badgering him to weigh in on this issue only.

At this point, though, I don't think I'm liable to add it retroactively,
because it would mean that existing ISession implementations would fall
out of compliance.

Note that there is no requirement currently that get_csrf_token store
the token as "_csrft_" in the session itself (or that the *_flash
methods store "_f_*" tokens). That's what the cookie-based
implementation does, but it's totally up to the ISession implementer as
to what they actually do, by design.

One compromise would be to add this function to pyramid.session:

def get_session_uuid(session):
""" Return the uuid associated with this session """
return session.setdefault('_uuid_', uuid4())

I'd consider doing that if it actually solved a concrete problem that
someone has already run into.

- C

[Randall Leeds]

Gotcha. I'm curious as well.

 

At this point, though, I don't think I'm liable to add it retroactively, because it would mean that existing ISession implementations would fall out of compliance.

Fair point.

 

Note that there is no requirement currently that get_csrf_token store the token  as "_csrft_" in the session itself (or that the *_flash methods store "_f_*" tokens).  That's what the cookie-based implementation does, but it's totally up to the ISession implementer as to what they actually do, by design.

That was rather my point.

 

One compromise would be to add this function to pyramid.session:

  def get_session_uuid(session):
      """ Return the uuid associated with this session """
      return session.setdefault('_uuid_',  uuid4())

Seems reasonable, but my first thought would be (request) and I could see it having a similar progression as pyramid.security.{authenticated_user,effective_principals}, moving to deprecation in favor of a request method. Maybe symmetry with security/auth is not a useful guide here, though. Clearly, a sane migration path would be needed, and you'd want to aim for consistency in APIs, whatever that means to you.

 

I'd consider doing that if it actually solved a concrete problem that someone has already run into.

+1 I'll bow out because I haven't one of my own. Only hypothetical ones.

[Jonathan Vanasco]

which is why I keep  badgering him to weigh in on this issue only.

sorry, i feel a bit clearer on what you want to hear now.

for this particular need, the others tabled, it's largely testing and troubleshooting -- and it is a bit more advanced.

things we've used that key for in the past and present:

   - unit and integrated tests that handle how session data is stored / persisted.  ensuring it is saved across requests on the backend store and validating the backend store directly.  it's a silly thing on one machine, but when you have a cluster of machines and start experimenting with Master/Slave replication, issues come up.   without exposing this identifier, one needs to trust some unseen "magic" that everything is being persisted correctly, and then do subsequent requests to see if data is being pulled out correctly.  i can't easily save a session and confirm how it has saved on memcached/redis ; I need to save a session and then load the session on a new request.  

  - this also simplifies tests that ensure we have loaded the correct session or unloaded/create a "new" one. the boolean 'new' loses utility if you invalidate/create multiple sessions in a single request.  chris' suggested `get_session_uuid` would handle this. 

  - this use case borders on a tabled discussion, but accessing the session_id allows us to more easily identify and load particular sessions, and create test cases around them.  for example, we may deploy some new code that stores/handles particular bits of session data differently... and that has broken some routes in the past.  by knowing the session_id, we can alter the actual backend data store to create "broken" sessions in the backend, and then use unit-tests / TDD to handle the issues gracefully.

   - admin / troubleshooting / support - being able to identify the internal id on a first request, so it can be cross-loaded into other requests ( not as a session, but as a raw data-stash pull ); we also have exception tweens that automatically log the exceptions,  session_id and a snapshot for later examination.  while an internal UUID can be logged , it can't be reliably used to 'load' data off the backend (session backends aren't necessarily full-text searchable; many are k/v pairs )

the other things we've used it for can be replaced by an "internal uuid" or a secondary cookie which is used as tracking identifier; however that does create the need to duplicate work and increase the amount of cookies or cookie storage.  being able to access the session_id allows for a lot of savings in workload and transmission -- but the big utility is in being able to log and confirm where data is saved to.

also, just to re-iterate, i'd be fine if this particular request weren't a "requirement" but a "suggested implementation".  if ISession doesn't support it natively, fine.  BUT if people build plugins that use an internal session key ,  there should be an official/recommended method on how to expose it.  sort of like dirty-needle programs for drug addicts: if you're going to do it, do it this way.

however - I don't think the potential drawbacks of the 'may return None'  situation is substantial. i can only see it being an issue in two scenarios :

a_ the cookie is client side

b_ the plugin hasn't supported the session_key yet.

There could be others, but those are the only ones i see.

if the cooke is client side, there is no session_key.  the programmer should never have accessed session_key.  if a session_id is needed, docs could instruct the usage of 'get_uuid' or creating an ancillary tracking cookie.

if the plugin hasn't supported the session_key yet, then -

1) there aren't many open source pyramid session packages.  patches and update to 90% would be relatively quick.

2) if there is a private session package, the developer would be calling a function that they haven't provided for yet.

@Randall-- for your use-case, I would probably use something like chris' suggestion -- where you have a set/get uuid function that accepts an explicit session.  we run 2 sessions simultaneously -- the normal `request.session` (which is an encrypted client-based-session) and a `request.session_https` that implements a server-side session locked down to https traffic ( it's just pyramid_beaker forked and bound to SSL and a different request attribute ).  we have code that operates on specific request attributes, but also other code that accepts and explicit session object.  if you have the potential for conflicts 

[Chris McDonough]

All of the above use cases presume that you have visibility into the
session implementation's backend to load data. If you do, that's fine,
but there are no APIs in Pyramid that provide this functionality, so
you're already in implementation-dependent mode when trying to satisfy
any of them. Having the ability to uniformly access a session's id
across ISession implementations buys you very little in these cases,
because you're already relying on implementation details that
necessarily go behind the back of ISession. You couldn't just swap out
one sessioning implementation for another, and expect these kinds of
tests or features to continue working, even if ISession did have an ID
API, right?

It seems to me that the id is least of your worries in each case, or at
least it would be a lot simpler to just depend on a custom concrete
ISession implementation than it would be to churn the gears to mandate
that all session implementations expose an id attribute. I say this
because, presuming we did mandate that, you'd still have a heck of a lot
of work to do if you wanted to switch sessioning implementations in
midstream to make those tests pass and features work.

> the other things we've used it for can be replaced by an "internal uuid"
> or a secondary cookie which is used as tracking identifier; however that
> does create the need to duplicate work and increase the amount of
> cookies or cookie storage. being able to access the session_id allows
> for a lot of savings in workload and transmission -- but the big utility
> is in being able to log and confirm where data is saved to.

Server-side sessions are usually quite expensive. Personally I use them
very sparingly. Unleash "ab" against a page that accesses session data
and see. Definitely if all I needed was some tracking identifier, I'd
use a plain old cookie, if only to reduce CPU cycles on the server side.
If you mean "I want to parse the session id out of the session cookie"
and not necessarily "I want to access the session", I'll note again that
the way cookies are composed is an detail of each ISession
implementation, so simply mandating that sessions have an ID attribute
doesn't buy you anything.

> also, just to re-iterate, i'd be fine if this particular request weren't
> a "requirement" but a "suggested implementation". if ISession doesn't
> support it natively, fine. BUT if people build plugins that use an
> internal session key , there should be an official/recommended method
> on how to expose it. sort of like dirty-needle programs for drug
> addicts: if you're going to do it, do it this way.
>
> however - I don't think the potential drawbacks of the 'may return None'
> situation is substantial. i can only see it being an issue in two
> scenarios :
>
> a_ the cookie is client side
> b_ the plugin hasn't supported the session_key yet.
>
> There could be others, but those are the only ones i see.

I already wrote details of a scenario where a None (optional) session id
is suboptimal in a prior email in this thread.

>
> if the cooke is client side, there is no session_key. the programmer
> should never have accessed session_key. if a session_id is needed, docs
> could instruct the usage of 'get_uuid' or creating an ancillary tracking
> cookie.
>
> if the plugin hasn't supported the session_key yet, then -
> 1) there aren't many open source pyramid session packages. patches and
> update to 90% would be relatively quick.
> 2) if there is a private session package, the developer would be calling
> a function that they haven't provided for yet.
>
>
> @Randall-- for your use-case, I would probably use something like chris'
> suggestion -- where you have a set/get uuid function that accepts an
> explicit session. we run 2 sessions simultaneously -- the normal
> `request.session` (which is an encrypted client-based-session) and a
> `request.session_https` that implements a server-side session locked
> down to https traffic ( it's just pyramid_beaker forked and bound to SSL
> and a different request attribute ). we have code that operates on
> specific request attributes, but also other code that accepts and
> explicit session object. if you have the potential for conflicts

In any case, without an actual in-the-wild use description of a required
feature that does not presume visibility into the session data backing
store or being able to parse the session id value out of a cookie, I'm
becoming more convinced that not requiring that an ISession implementer
expose the session id uniformly is actually correct. That doesn't mean
that session implementations can't expose it; pyramid_redis_session
sessions already expose the session id (as session.session_id, FWIW), so
you could rely on this implementation detail as easily as you're already
relying on the detail (at least if you use pyramid_redis_sessions) that
session data gets stored in redis. Relying on this would be no worse
than writing integration tests that check redis to see if the data
actually winds up there.

- C

[Jonathan Vanasco]

On Thursday, May 8, 2014 7:16:43 AM UTC-4, Chris McDonough wrote:

All of the above use cases presume that you have visibility into the
session implementation's backend to load data.  If you do, that's fine,
but there are no APIs in Pyramid that provide this functionality, so
you're already in implementation-dependent mode when trying to satisfy
any of them.  

Agreed.  I'm not trying to use this data to do anything "with" the Pyramid system. 

Having the ability to uniformly access a session's id
across ISession implementations buys you very little in these cases,
because you're already relying on implementation details that
necessarily go behind the back of ISession.  You couldn't just swap out
one sessioning implementation for another, and expect these kinds of
tests or features to continue working, even if ISession did have an ID
API, right?

Yes and No. 

Incompatible:  If I were switching out a Redis for a Memcached system, then I'd need to have tests that can adapt the "loading" mechanism or are specific to whatever is active.

Compatible: If I were switching between two Redis or two Memcached backends, then everything should work.

 

Server-side sessions are usually quite expensive.  Personally I use them
very sparingly.  Unleash "ab" against a page that accesses session data
and see. Definitely if all I needed was some tracking identifier, I'd
use a plain old cookie, if only to reduce CPU cycles on the server side.

Agreed.  That's why we use 2 sessions -- encrypted cookie on http ; server side on https -- and that's why we've switched backends a bunch, trying to edge out a little bit more performance.  Our https session data has data in it that we don't want to expose publicly (even in encrypted form) and is expensive/large to load.  So it's more efficient in this case.

In any case, without an actual in-the-wild use description of a required
feature that does not presume visibility into the session data backing
store or being able to parse the session id value out of a cookie, I'm
becoming more convinced that not requiring that an ISession implementer
expose the session id uniformly is actually correct.  That doesn't mean
that session implementations can't expose it; pyramid_redis_session
sessions already expose the session id (as session.session_id, FWIW), so
you could rely on this implementation detail as easily as you're already
relying on the detail (at least if you use pyramid_redis_sessions) that
session data gets stored in redis.  Relying on this would be no worse
than writing integration tests that check redis to see if the data
actually winds up there.

Being an implementation detail is fine and fair.  As I said before, I'm more concerned with the other topic, which would also handle any of my needs.

I still would really like to see a "Recommended Practice" in the docs so that, if another person builds a session plugin and decides to expose this data, it would be the same as pyramid_redis_session or something else.  

[Mike Orr]

On Thu, May 8, 2014 at 4:16 AM, Chris McDonough <chr...@plope.com> wrote:
> All of the above use cases presume that you have visibility into the session
> implementation's backend to load data. If you do, that's fine, but there
> are no APIs in Pyramid that provide this functionality, so you're already in
> implementation-dependent mode when trying to satisfy any of them.

That's the point of this discussion: the API is missing a pretty
standard feature.

> One compromise would be to add this function to pyramid.session:
>
> def get_session_uuid(session):
> """ Return the uuid associated with this session """
> return session.setdefault('_uuid_', uuid4())

I'm not sure about that as a method, but another compromise would be
to document the pattern (i.e., the expression) in the session
documentation. That would give users something they could drop into
their code, and avoid reinventing it or searching the session source
code in vain.

> It seems to me that the id is least of your worries in each case, or at
> least it would be a lot simpler to just depend on a custom concrete ISession
> implementation than it would be to churn the gears to mandate that all
> session implementations expose an id attribute.

That puts the burden on the user! Not all users can figure out how to
make a working ISession implementation or modify
'pyramid_redis_sessions', and it takes a lot of time.

I don't see why you consider it more acceptable to force this on users
than to just modify the existing ISession implementations in one
swoop. There are only two of them I'm aware of, 'pyramid_beaker' and
'pyramid_redis_sessions'. 'pyramid_beaker' already has 'session.id',
so it's just a matter of getting it into 'pyramid_redis_sessions'.
Which already has the ID, it just needs to put it on 'session.id'. The
implementations in Pyramid would all use the dummy alternative (i.e.,
None) since they don't have IDs. So 1, 2, 3 and it's done. The hardest
part is 'pyramid_redis_sessions', but that would be easier for the
maintainer who knows the code to do than for a user to do (because I
tried and failed).

> I say this because,
> presuming we did mandate that, you'd still have a heck of a lot of work to
> do if you wanted to switch sessioning implementations in midstream to make
> those tests pass and features work.

I wouldn't expect existing sessions to remain valid when switching
backends. I'd clear them all out.

> I already wrote details of a scenario where a None (optional) session id is
> suboptimal in a prior email in this thread.

It's a theoretical use case, like this three-independent-person
scenario you're insisting in. Whereas Jonathan and I have concrete
existing cases.

> session id uniformly is actually correct. That doesn't mean that session
> implementations can't expose it; pyramid_redis_session sessions already
> expose the session id (as session.session_id, FWIW),

No it doesn't. That's how I lost a month's worth of session counts
when I switched from 'pyramid_beaker' to 'pyramid_redis_sessions'.

--
Mike Orr <slugg...@gmail.com>

[Chris McDonough]

On 05/08/2014 01:05 PM, Jonathan Vanasco wrote:
> Having the ability to uniformly access a session's id
> across ISession implementations buys you very little in these cases,
> because you're already relying on implementation details that
> necessarily go behind the back of ISession. You couldn't just swap out
> one sessioning implementation for another, and expect these kinds of
> tests or features to continue working, even if ISession did have an ID
> API, right?
>
>
> Yes and No.
>
> Incompatible: If I were switching out a Redis for a Memcached system,
> then I'd need to have tests that can adapt the "loading" mechanism or
> are specific to whatever is active.
>
> Compatible: If I were switching between two Redis or two Memcached
> backends, then everything should work.

I'd be surprised. That'd presume the two implementations stored data in
the exact same way, which might well be the case, but can't be assumed.
If it didn't break, it'd only be by happy accident.

The other topic, at this point, is making it possible to uniformly parse
the session id out of the session cookie?

> I still would really like to see a "Recommended Practice" in the docs so
> that, if another person builds a session plugin and decides to expose
> this data, it would be the same as pyramid_redis_session or something else.

I'm still not sure how that would actually help you as much as you think
it would, given that if you change the session implementation, you will
already necessarily need to change code, or at least there's no
guarantee that things will "just work" when you swap in a random session
implementation given your description of the cases you want to support.

In the meantime, I'm pretty loath to make "interface suggestions": APIs
are really where the rubber meets the road. It's either in the
interface or it's not, and I'm leaning towards not.

That doesn't mean that you are out of luck, though. Have you thought of
this? It would require that you change code when you change session
implementations, but you already have to do that anyway.

# in my_package

from pyramid_redis_sessions import RedisSessionFactory
from pyramid.session import SignedCookieSessionFactory

def my_session_factory(**settings):
cookie_session_factory = SignedCookieSessionFactory(**settings)
redis_session_factory = RedisSessionFactory(**settings)
def session_factory(request):
if request.scheme == 'http':
session = cookie_session_factory(request)
session.id = None
else:
session = redis_session_factory(request)
session.id = session.session_id
return session
return session_factory

# in configuration code

from my_package import my_session_factory

settings = {...}
config.set_session_factory(my_session_factory(**settings))

<Some massaging required to pass "the right" settings to each factory
you're delegating to left out....>

This would give you a single place to make any changes necessary to the
session before returning it. You would effectively be reusing existing
session factory implementations and massaging the result of each to
provide a locally defined session interface. Composition is magic when
applied this way.

- C

[Chris McDonough]

On 05/08/2014 01:21 PM, Mike Orr wrote:
> On Thu, May 8, 2014 at 4:16 AM, Chris McDonough <chr...@plope.com> wrote:
>> All of the above use cases presume that you have visibility into the session
>> implementation's backend to load data. If you do, that's fine, but there
>> are no APIs in Pyramid that provide this functionality, so you're already in
>> implementation-dependent mode when trying to satisfy any of them.
>
> That's the point of this discussion: the API is missing a pretty
> standard feature.

That may be true, but I'm sure you can see that session objects having a
uniform .id attribute won't solve the entirety of the problem implied by
the above complaint.

If you mean that being able to load session data using a session id is a
required feature, I'm pretty sure that ISession is not missing a
feature. You need to have a session object to use the ISession
interface, and you obviously can't get one without loading a session. So
when you say something like the above, I think you're really asking for
an additional interfaces e.g.

def session_id_from_request(request):
""" Return the session id of the current request """

def load_session(request, session_id):
""" Return the session object given the provided session id """

The conversation started by you and Jonathan insisting on "just" having
a uniform session id attribute on an already-loaded session object, but
AFAICT, and given everyone's descriptions of their problems, just having
that feature it just doesn't seem to make much difference in isolation.
The reasons given for wanting it seem to always imply wanting the
above APIs as well, or at least Jonathan hasn't provided any cases where
one or the other is not ultimately necessary to not need to read the
source code of the implementation.

Given your use cases, does it make maximum sense to have a uniform id
attribute on the session without having these APIs? If so, please help
me figure out how. But if we can't think of a compelling reason to have
session.id in isolation from something like the above APIs, then we have
to consider adding them as well. I'm pretty uninterested in adding
features to the interface that doesn't actually solve the real problem,
or leaves you needing to read the source code of each session
implementation anyway in order to go behind its back and read cookie
data from the request or session data from the backing store. That'd be
a half-solution that would only add confusion.

>> One compromise would be to add this function to pyramid.session:
>>
>> def get_session_uuid(session):
>> """ Return the uuid associated with this session """
>> return session.setdefault('_uuid_', uuid4())
>
> I'm not sure about that as a method, but another compromise would be
> to document the pattern (i.e., the expression) in the session
> documentation. That would give users something they could drop into
> their code, and avoid reinventing it or searching the session source
> code in vain.
>
>> It seems to me that the id is least of your worries in each case, or at
>> least it would be a lot simpler to just depend on a custom concrete ISession
>> implementation than it would be to churn the gears to mandate that all
>> session implementations expose an id attribute.
>
> That puts the burden on the user! Not all users can figure out how to
> make a working ISession implementation or modify
> 'pyramid_redis_sessions', and it takes a lot of time.

You're willing to read pyramid_redis_sessions to figure out how to parse
its cookie and/or how to get data out of its backing store, and you
don't think that relying on its implementation details in this way is a
problem, but reading its code to figure out where it stashes the session
id is infuriating?

> I don't see why you consider it more acceptable to force this on users
> than to just modify the existing ISession implementations in one
> swoop. There are only two of them I'm aware of, 'pyramid_beaker' and
> 'pyramid_redis_sessions'. 'pyramid_beaker' already has 'session.id',
> so it's just a matter of getting it into 'pyramid_redis_sessions'.
> Which already has the ID, it just needs to put it on 'session.id'. The
> implementations in Pyramid would all use the dummy alternative (i.e.,
> None) since they don't have IDs. So 1, 2, 3 and it's done. The hardest
> part is 'pyramid_redis_sessions', but that would be easier for the
> maintainer who knows the code to do than for a user to do (because I
> tried and failed).
>
>> I say this because,
>> presuming we did mandate that, you'd still have a heck of a lot of work to
>> do if you wanted to switch sessioning implementations in midstream to make
>> those tests pass and features work.
>
> I wouldn't expect existing sessions to remain valid when switching
> backends. I'd clear them all out.

I'm not talking about session data. I'm talking about code that
Jonathan wrote to verify that indeed session info was *put into redis*.
That was the case he gave. If he switched sessioning implementations,
he's going to need to change that code. Likewise, if he goes behind the
back of ISession to "cross-load session data" as he described, he's
going to need to change not just test code, but runtime code too.

>> I already wrote details of a scenario where a None (optional) session id is
>> suboptimal in a prior email in this thread.
>
> It's a theoretical use case, like this three-independent-person
> scenario you're insisting in. Whereas Jonathan and I have concrete
> existing cases.

You do indeed have concrete existing use cases, but they aren't
satisfied by just adding a session.id, AFAICT.

>> session id uniformly is actually correct. That doesn't mean that session
>> implementations can't expose it; pyramid_redis_session sessions already
>> expose the session id (as session.session_id, FWIW),
>
> No it doesn't. That's how I lost a month's worth of session counts
> when I switched from 'pyramid_beaker' to 'pyramid_redis_sessions'.

I'm not sure how to interpret this, then:
<https://github.com/ericrasmussen/pyramid_redis_sessions/blob/master/pyramid_redis_sessions/session.py#L74>

- C

[Jonathan Vanasco]

On Thursday, May 8, 2014 1:40:58 PM UTC-4, Chris McDonough wrote:

I'd be surprised.  That'd presume the two implementations stored data in
the exact same way, which might well be the case, but can't be assumed.
  If it didn't break, it'd only be by happy accident.

agreed.  most just pickle a dict.  a few wrap the dict in something.  but most just pickle a dict and have internal '__' prefixed fields

 

The other topic, at this point, is making it possible to uniformly parse
the session id out of the session cookie?

For the sake of clarity:

- The other topic is to tie the "session's cookie" to the "session object" itself.

- This would have the benefit of *potentially* making it possible to parse the session_id out of the cookie.  That would take additional development effort by the end consumer, as different session schemes encode the session id in a cookie differently.  Depending on the scheme, this might not be easy/possible.

- This would not be as convenient as being able to access a session_id, but it would allow for actual sessions to be identified and trackable and provide for accountability in testing.

- This would remove the "black box magic" of the current API, where it is impossible to accurately tell where / how a session was created.

I'm still not sure how that would actually help you as much as you think
it would, given that if you change the session implementation, you will
already necessarily need to change code, or at least there's no
guarantee that things will "just work" when you swap in a random session
implementation given your description of the cases you want to support. 

In the meantime, I'm pretty loath to make "interface suggestions": APIs
are really where the rubber meets the road.  It's either in the
interface or it's not, and I'm leaning towards not.

I get that.  And I don't expect to sway you on this.  But this is where Mike and I bring up that pesky point that all server side sessions feature this... somehow.

Without any guidance, they're implemented differently.  The needs to access this id may be specialized, but are common enough.

Instead of standardizing how a common feature / implementation detail takes place, things are left to be wild.

The "Pro" of supporting an official API point is that developers would have guidance and something to write upon.  There would be some sort of consistency that leads to higher interoperability.

The "Cons" seem to be focus on:

- Potential issues/constraints that future session developers might encounter

- The potential to return "None" (or raise a NotImplementedError) if an application developer accesses this in an unsupported environment.  

The first argument is very weak to me, especially on the benefit.  It seems be a theoretical issue, not a practical one.

I still simply don't understand any use cases where the second argument would actually be a problem.  That wouldn't be much different from someone trying to commit a transaction in a transactionless environment.  

That doesn't mean that you are out of luck, though.  Have you thought of
this?  It would require that you change code when you change session
implementations, but you already have to do that anyway.

i think we used a NewRequest subscriber to map a session_id attribute onto the session.  I'd have to dig into code.

we run `request.session` and `request.session_https` ; this allows https calls to read/set the non-https session data.  a client side cookie would never interact with a session_id, so that functionality is not needed.

Also, the biggest problem of your approach -- and mine -- isn't that I need to update my code.  It's that I need to go through all of the session plugin's code to figure out how the key is being managed.   i end up reinventing the wheel a lot on this; as does mike. i'm sure others do too.  instead of doing all these one-off fixes, i'd rather fork the plugin, make it compatible with an API option and then issue a pull request so others can enjoy benefit of my work.

[Chris McDonough]

On 05/08/2014 04:42 PM, Jonathan Vanasco wrote:
>
>
> On Thursday, May 8, 2014 1:40:58 PM UTC-4, Chris McDonough wrote:
>
> I'd be surprised. That'd presume the two implementations stored
> data in
> the exact same way, which might well be the case, but can't be assumed.
> If it didn't break, it'd only be by happy accident.
>
>
> agreed. most just pickle a dict. a few wrap the dict in something.
> but most just pickle a dict and have internal '__' prefixed fields

By "most" I assume you mean Beaker, because pyramid_redis_sessions
doesn't put any __ prefixed fields into its session data. There's no
guarantee about any of that stuff, it's 100% implementation dependent.

> The other topic, at this point, is making it possible to uniformly
> parse
> the session id out of the session cookie?
>
>
> For the sake of clarity:
>
> - The other topic is to tie the "session's cookie" to the "session
> object" itself.
> - This would have the benefit of *potentially* making it possible to
> parse the session_id out of the cookie. That would take additional
> development effort by the end consumer, as different session schemes
> encode the session id in a cookie differently. Depending on the scheme,
> this might not be easy/possible.
> - This would not be as convenient as being able to access a session_id,
> but it would allow for actual sessions to be identified and trackable
> and provide for accountability in testing.
> - This would remove the "black box magic" of the current API, where it
> is impossible to accurately tell where / how a session was created.

I understand this to mean (from your earlier emails, you don't expressly
say it above) that you want to be sure that there is a existing session
that has the id "123" if you parse "123" out of the cookie. Since we
can't stop browsers from sending cookie data, and since sessions have a
timeout (and as a result even a valid session id might not resolve to a
non-new session), I don't see how it would be possible to make that
promise due to the way the web works. The only implementation that
would have a shot at it would be purely cookie-value-based sessions,
because the value is stored in the cookie itself.

If that's not what you mean, can you explain the difference between what
you're asking for and "I want to be able to get the session id using
only request data, without loading the session"? Having an .id on
session objects would not help with that at all, AFAICT.

In any case, wrt black box magic, I'm sure you understand that the point
of an interface is to create a black box, such that one box can be
swapped for another by otherwise unconcerned consumers without the
consumer needing to open either box. Once you start dithering in the
interface definition about what's expected of implementations, it's
understandable that those implementations aren't going to be able to
satisfy the intent of the interface. In general, "optional" APIs, APIs
which are permitted to raise a NotImplemented exception or APIs which
return a "sorry Dave I can't do that" sentinel are usually indicators of
an interface that isn't very well defined. I know this because I've
perpetrated plenty in my time.

Committing a transaction doesn't return a value. The person doing the
committing doesn't need to inspect the return value of a call which
commits a transaction, so committing a transaction in a transactionless
system can be a noop; something called purely for its side effects
(which may be no side effects at all). Asking for a session id and
getting back None is a different, because the caller cares deeply about
the return value; he wouldn't ask for it if he weren't going to use it
for something.

Since I already wrote down the problems with the value sometimes being
None I won't bore you with them again, but note that this example gets
to the crux of the issue. If you need to access the "real" session id
(as opposed to some UUID stored inside session data), it necessarily
means you intend to do something useful with it. There are currently no
APIs in Pyramid that allow you to do anything useful with it at all, so
by definition you're always going to use it "behind the back" of any
interface. Even if you just log it, presumably the reason you care
deeply that it's the "real" session id is to be able to later make some
sort of postmortem query of the session storage to debug something,
right? Otherwise you wouldn't much care about it being the same key as
the key used by the backing store, and you could just use a UUID stored
as session data.

I don't care at all if you use the "real" session id for whatever you
like, it's totally fine by me. But I can't wrap my brain around why
it's such a burden to need to depend on session plugin implementation
details to get the "real" session id and it's seemingly such a lesser
burden to do need to depend on those same plugin implementation details
to do something useful with that same session id. If the argument,
instead, was "I want to be able to load session data using an arbitrary
session id via a well-defined interface so I don't have to go look at
any implementation details", I'd understand. But nobody has asked for
that so far. Is it because it seems like a bridge too far, or what?

To truly avoid you needing to read any plugin code, we'd need session
add-ons to provide implementations for APIs like I described in my last
emails to Mike:

def session_id_from_request(request):
""" Return the session id of the current request """

def load_session(request, session_id):
""" Return the session object given the provided session id """

> That doesn't mean that you are out of luck, though. Have you
> thought of
> this? It would require that you change code when you change session
> implementations, but you already have to do that anyway.
>
>
> i think we used a NewRequest subscriber to map a session_id attribute
> onto the session. I'd have to dig into code.

FWIW, this means that each request, even one for a static resource like
an image or a JS file, is potentially creating a (or at least
deserializing an existing) session object. This is almost sure to be a
bottleneck and may be slowing down your app by an order of magnitude or
two. Deferring until the session is accessed via its factory (when
request.session is accessed) prevents this from happening, so it's
probably better to do it that way.

> we run `request.session` and `request.session_https` ; this allows https
> calls to read/set the non-https session data. a client side cookie
> would never interact with a session_id, so that functionality is not needed.
>
> Also, the biggest problem of your approach -- and mine -- isn't that I
> need to update my code. It's that I need to go through all of the
> session plugin's code to figure out how the key is being managed. i
> end up reinventing the wheel a lot on this; as does mike. i'm sure
> others do too. instead of doing all these one-off fixes, i'd rather
> fork the plugin, make it compatible with an API option and then issue a
> pull request so others can enjoy benefit of my work.

When you say "how the key is being managed", you mean figuring out what
attribute of the session that the key into the backing store is being
set into? Does this actually take a lot of time as compared to all the
time it takes to figure out how the session machinery stores its data so
you can yank it out later? If so, please describe the wheels you need
to reinvent here and one-off fixes you need to do. Does my composition
example give you any ideas about how to fix it without patching anything?

- C

[Jonathan Vanasco]

On Thursday, May 8, 2014 8:08:37 PM UTC-4, Chris McDonough wrote:

On 05/08/2014 04:42 PM, Jonathan Vanasco wrote:

 

I understand this to mean (from your earlier emails, you don't expressly
say it above) that you want to be sure that there is a existing session
that has the id "123" if you parse "123" out of the cookie.  Since we
can't stop browsers from sending cookie data, and since sessions have a
timeout (and as a result even a valid session id might not resolve to a
non-new session), I don't see how it would be possible to make that
promise due to the way the web works.  The only implementation that
would have a shot at it would be purely cookie-value-based sessions,
because the value is stored in the cookie itself.

No.  My desire is to associate the session to an incoming value so that I can do things like

- have full transparency on how / why the session is created 

- replay requests during troubleshooting

- simplify integrated tests that involve cookie data

deriving session_ids and other stuff out of a cookie value is possible, but not guaranteed.  while i could easily do it with current session libraries, i can't and rely on that.

as i explained above, and showed in a gist, there is no direct connection between a Session and the cookie it is related to ( or other possible 'public identifier' ).  In the case of cookies, this value could exist on the Request or Response object (server side cookie) and request object (client side cookie).   

i can assume that a "not new" session should correlate to an incoming cookie; and a "new" session should correlate to an outgoing cookie.  however there is no certainty here; it's an assumption.  being able to associate an input with a session is largely understanding an implementation detail.

i am not concerned with the data serialized inside the cookie, or associated to it via a backend store ( directly or though a cookie encoded value ).  i'm just concerned with being able to correlate an input value (cookie) to a session and vice-vesra.

I also understand that developers will care deeply about a return value; however they shouldn't be expecting/caring for one that doesn't exist by virtue of their choice in session stores.  I'm not disagreeing with your assessment, I just see it more like a paradox.

In any case, wrt black box magic, I'm sure you understand that the point
of an interface is to create a black box, such that one box can be
swapped for another by otherwise unconcerned consumers without the
consumer needing to open either box.  Once you start dithering in the
interface definition about what's expected of implementations, it's
understandable that those implementations aren't going to be able to
satisfy the intent of the interface.

Yes, but there is a divide between systems that operate with "total magic" and keep most things away from developers ( Django, Rails) and systems that don't hide the underpinnings ( Pylons, Pyramid).  The way that Pyramid functions -- which extends beyond the ISession interface -- sessions are a black box.  They sort of magically work -- which is great -- but you can't access any of the HOW or WHY unless you write custom session implementations.

If the argument, instead, was "I want to be able to load session data using an arbitrary session id via a well-defined interface so I don't have to go look at  any implementation details", I'd understand.  But nobody has asked for that so far.  Is it because it seems like a bridge too far, or what? 

Personally, I'd love to have something work like that. An ISessionLoader or similar.  It would make testing so much easier, as we do that already :)   But if there is a huge debate over getting just ID, it doesn't seem worth bringing up.

 

> FWIW, this means that each request, even one for a static resource like an image or a JS file, is potentially creating a (or at least deserializing an existing) session object.  This is almost sure to be a bottleneck and may be slowing down your app by an order of magnitude or two.  Deferring until the session is accessed via its factory (when request.session is accessed) prevents this from happening, so it's  probably better to do it that way. 

Only dynamic content hits Pyramid in our stack; nginx/varnish catches the rest.  Sessions are hit on every page to pull out "is_logged_in" and other data.  Since most data got moved into a read-through cache, our biggest bottlenecks are actually template rendering (!).  

> When you say "how the key is being managed", you mean figuring out what attribute of the session that the key into the backing store is being
set into?  Does this actually take a lot of time as compared to all the time it takes to figure out how the session machinery stores its data so
you can yank it out later? 

Actually, yes.  The data serialization tends to be straightforward.  It usually takes a couple seconds to identify.  It's often simple.

The session key stuff tends to be harder to find, as the "raw" id goes by different names and can be somehow encoded.  

We do a lot more tracking with the raw key than the tracked session data though.