Session

[elin]

There have been a long series of issues, email threads, and forum reports of issues involving sessions.  These are not simple and not all completely platform related but they all ultimately have platform implications. Unfortunately for me I also just reread about 10 CMS tracker issues involving this. So I think all of us, whether working on the CMS, the platform or extensions need some coherent guidance about how to tackle these issues in a way that makes sense overall instead of fixing symptoms. 

Here are the basic issues as I understand them.

1. There is a problem with session handling in mysqli that is addressed by this pull request https://github.com/elinw/joomla-platform/commit/e441e589b35896732ad31ec03688a5a3d3a1fd31 . Given that the cms now defaults to mysqli this may explain a number of problems.

2. There are major issues with using anything besides database for session. (You can also search for memcache in the tracker to see some other reports).

3. It has been proposed that the CMS joomla.sql file be modified to make the datafield LONGTEXT and the session ID VARCAR(200), which is of course not a platform issue, but shouldn't the library fail gracefully if the data get too big?  No matter how big a field an application has sooner or later something is going to push it too far. This MAY explain some of the cases where we are seeing duplicate session records created (assuming they are not caused by #1).

4. Ofer reported another issue that I don't think any of us have totally run down yet (you need to read his issue in between Adrien's).

5. There are numerous todos regarding raising errors and warnings in session.php. Currently these all reference using JError but at this point I think there needs to be some guidance about whether these are places where things should be logged or if there should be exceptions thrown. If logged, what category and level? 

6. There is a problem sometimes with "too many cookies" (and many duplicates) causing the session to be ended.

7. There are a bunch of other issues also, but my sense is some of them may resolve once the above are dealt with.

Whenever there are so many issues in one file there are going to be be problems with file conflicts not to mention, as I said, focus on fixing symptoms rather than root causes. So  I'm wondering if there is a way to proceed with addressing the platform side of these things in a way that minimizes that.  Maybe one person could coordinate? Is anyone looking at this already?

Elin

[Ofer Cohen]

Hey Elin,

I've change recently to mysql (instead of mysqli) and the problem did not get back. So maybe there is relation between my issue  to the mysqli issue.

Best Regards,

Ofer Cohen

--
You received this message because you are subscribed to the Google Groups "Joomla! Framework Development" group.
To view this discussion on the web, visit https://groups.google.com/d/msg/joomla-dev-framework/-/bv58dNMxV2AJ.
To post to this group, send an email to joomla-dev...@googlegroups.com.
To unsubscribe from this group, send email to joomla-dev-frame...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/joomla-dev-framework?hl=en-GB.

[Lobos]

I have mentioned elsewhere that there seems to be a problem with
JSession and automatic login (ie for remember me) - say if you set
session data via JSession it will be lost with the automatic login.
This to me seems counter-intuitive as one would think the session
lifetime would be linked to the user's login state... not sure really
if we are meant to create our own session type functionality in
components... To me it would seem better if JSession retained any
session data set and not wipe it out at login or login, but even that
sounds strange... so bigggggggg IDK! hahahhaha... for example imagine
if you developing a shopping cart - you would expect the shopping cart
contents to persist throughout login, logout, etc as a guest might
visit the site, add some items to their cart and then register and
then login - the cart contents need to persist. Then you don't want
the user to lose their cart if they happen to get interrupted and
leave the computer for a few hours as the session is wiped on auto
logout - even if they have checked "Remember Me" during login the
session is still lost at the auto login that Remember Me invokes...

On 15 Aug, 09:36, Ofer Cohen <oferc...@gmail.com> wrote:
> Hey Elin,
>
> I've change recently to mysql (instead of mysqli) and the problem did not get back. So maybe there is relation between my issue  to the mysqli issue.
>

> Best Regards,Ofer Cohen

> On 08/14/2011 09:40 PM, elin wrote:There have been a long series of issues, email threads, and forum reports of issues involving sessions.  These are not simple and not all completely platform related but they all ultimately have platform implications. Unfortunately for me I also just reread about 10 CMS tracker issues involving this. So I think all of us, whether working on the CMS, the platform or extensions need some coherent guidance about how to tackle these issues in a way that makes sense overall instead of fixing symptoms. 
>
> Here are the basic issues as I understand them.
>
>
>
> 1. There is a problem with session handling in mysqli that is addressed by this pull request https://github.com/elinw/joomla-platform/commit/e441e589b35896732ad31ec03688a5a3d3a1fd31 . Given that the cms now defaults to mysqli this may explain a number of problems.
>
>
>

> 2. There are major issues with using anything besidesdatabase for session. (You can also search for memcache in the tracker to see some other reports).

>
>
>
> 3. It has been proposed that the CMS joomla.sql file be modified to make the datafield LONGTEXT and the session ID VARCAR(200), which is of course not a platform issue, but shouldn't the library fail gracefully if the data get too big?  No matter how big a field an application has sooner or later something is going to push it too far. This MAY explain some of the cases where we are seeing duplicate session records created (assuming they are not caused by #1).
>
>
>

> 4. Ofer reported another issue that I don't think any of us have totallyrun down yet (you need to read his issue in between Adrien's).

>
>
>
> 5. There are numerous todos regarding raising errors and warnings in session.php. Currently these all reference using JError but at this point I think there needs to be some guidance about whether these are places where things should be logged or if there should be exceptions thrown. If logged, what category and level? 
>
>
>
> 6. There is a problem sometimes with"too many cookies" (and many duplicates) causing the session to be ended.
>
>
>
> 7. There are a bunch of other issues also, but my sense is some of them may resolve once the above are dealt with.
>
>
>
> Whenever there are so many issues in one file there are going to be be problems with file conflicts not to mention, as I said, focus on fixing symptoms rather than root causes. So  I'm wondering if there is a way to proceed with addressing the platform side of these things in a way that minimizes that.  Maybe one person could coordinate? Is anyone looking at this already?
>
>
>
> Elin
>
> --
> You received this message because you are subscribed to the Google Groups "Joomla! Framework Development" group.

> To view this discussion on the web, visithttps://groups.google.com/d/msg/joomla-dev-framework/-/bv58dNMxV2AJ.
> To post to this group, send an email tojoomla-de...@googlegroups.com.
> To unsubscribe from this group, send email tojoomla-dev-fram...@googlegroups.com.

[Andrew Eddie]

On 15 August 2011 04:40, elin <elin....@gmail.com> wrote:
> 1. There is a problem with session handling in mysqli that is addressed by
> this pull
> request https://github.com/elinw/joomla-platform/commit/e441e589b35896732ad31ec03688a5a3d3a1fd31 .
> Given that the cms now defaults to mysqli this may explain a number of
> problems.

We can't allow a database driver to close the session. Apart from the
fact that it's not good design, some apps could be using multiple
database connections and destroying one will destroy the session. I'm
not sure what the solution is (given reproducing the problem is
elusive), but that isn't it.

> 3. It has been proposed that the CMS joomla.sql file be modified to make the
> datafield LONGTEXT and the session ID VARCAR(200), which is of course not a
> platform issue, but shouldn't the library fail gracefully if the data get
> too big?  No matter how big a field an application has sooner or later
> something is going to push it too far. This MAY explain some of the cases
> where we are seeing duplicate session records created (assuming they are not
> caused by #1).

I'm not sure why the session id needs to be increased, but allowing
for session data of 4Gb for each session seems a bit over the top.
MEDIUMTEXT is 16Mb which should be adequate. If it's not, then
developers, or a combination of a number of heavy extensions, are
simply pushing too much information into the session.

> 6. There is a problem sometimes with "too many cookies" (and many
> duplicates) causing the session to be ended.

That's caused by the sliders. Turn off the remember open position
(except for possibly Global Configuration) and this should go away.

Regards,
Andrew Eddie

>
> --
> You received this message because you are subscribed to the Google Groups
> "Joomla! Framework Development" group.
> To view this discussion on the web, visit
> https://groups.google.com/d/msg/joomla-dev-framework/-/bv58dNMxV2AJ.
> To post to this group, send an email to

> joomla-dev...@googlegroups.com.

> To unsubscribe from this group, send email to

> joomla-dev-frame...@googlegroups.com.

[Andrew Eddie]

Lobos, that feature simply doesn't exist so someone would have to work
out how to do it.

Regards,
Andrew Eddie
http://learn.theartofjoomla.com - training videos for Joomla 1.6 developers

> To post to this group, send an email to joomla-dev...@googlegroups.com.
> To unsubscribe from this group, send email to joomla-dev-frame...@googlegroups.com.

[Nicholas K. Dionysopoulos]

Yes, 16Mb should be adequate. 4Gb was a quick and dirty solution, but given the usual PHP memory limits even 16Mb is spacious for all practical intents and purposes :)

Nicholas K. Dionysopoulos, lead developer AkeebaBackup.com

[Louis Landry]

Hi,

What you are describing here to me doesn't sound like a session use-case at all.  It seems to me like data that should be tied to the user, not the session.  You'd be better off IMO handling a cookie per user and mapping that to a cart in a cart table(s) outside of the session which handles authentication/security.  Especially since you are likely going to want to have relational information about the data in people's carts.  What if the marketing manager for that company decided they wanted to know what the top 5 products were that were in people's carts but not yet purchased?  

I certainly understand the sentiment of feeling like 2k of data is too small for a session (though IMO if you are doing it right it really isn't), but there is absolutely no need for a huge data store there.  It is trying to put a round peg in a square hole.

- Louis

On 19 August 2011 03:23, Lobos <adam.d...@gmail.com> wrote:

I have mentioned elsewhere that there seems to be a problem with

JSession and automatic login (ie for remember me) - say if you set

session data via JSession it will be lost with the automatic login.

This to me seems counter-intuitive as one would think the session

lifetime would be linked to the user's login state... not sure really

if we are meant to create our own session type functionality in

components... To me it would seem better if JSession retained any

session data set and not wipe it out at login or login, but even that

sounds strange... so bigggggggg IDK! hahahhaha... for example imagine

if you developing a shopping cart - you would expect the shopping cart

contents to persist throughout login, logout, etc as a guest might

visit the site, add some items to their cart and then register and

then login - the cart contents need to persist. Then you don't want

the user to lose their cart if they happen to get interrupted and

leave the computer for a few hours as the session is wiped on auto

logout - even if they have checked "Remember Me" during login the

session is still lost at the auto login that Remember Me invokes...

------------------------------------------------------------

Louis Landry

louis....@gmail.com

[Andy Nagai]

how about create a cookie with an encrypted cart id that persists
weather or not they are logged in.

> To post to this group, send an email to joomla-dev...@googlegroups.com.
> To unsubscribe from this group, send email to joomla-dev-frame...@googlegroups.com.

[Nicholas Dionysopoulos]

Hi Louis,

Carts are certainly the worst use case for sessions. But here's a legitimate use case where 20K is hardly enough. Traversing the filesystem tree, changing permissions using JFTP on a typical shared server. The process is ad slow and you wouldn't want the whole process to time out. As a result you need to split the process in many parts. Each part runs on a single page load. Since PHP is stateless you need to keep the current directory and file list stack somewhere. This somewhere seems to be the session. All is good until you hit that folder 3 levels deep with 1000 files in it, I mean images/stories. Storing the full pathnames of 1k files busts the session storage and your code is busted. That's what happened to me with Admin Tools.

So, no, 20kb of session storage is not enough any more than 640kb of RAM ought to be enough for everyone ;)

Nicholas K. Dionysopoulos

Lead Developer, AkeebaBackup.com

Web: https://www.akeebabackup.com

Blog: http://www.dionysopoulos.me/blog

Sent from my iPad. Please excuse my brevity.

[Louis Landry]

I understand your point Nic, though still in that case I don't think the session is the most appropriate place (though admittedly the easiest) to store that information.  I mean in a perfect world I'd probably look at encapsulating each of those larger operations as a job with a job id.  Then probably store that in a db table or at least some metadata about it like start/end times, etc.  That way you can report on it later or something.  If it is added into session data then you won't ever have data to look at later.

I mean, look, I get it.  I totally get it, but in these cases we aren't *exactly* talking about something that needs to be tied to a user's session.  In fact, I could think of a few things about these each of these examples that should/could exist outside the scope of a user's logged in session.  Please no one take it as me saying that we have to keep the session table's data field like it is.  I didn't set it to what it is, and I don't have a horse in that race.  I do think 20k is a lot of storage space for the incidental stuff I personally see as worth putting into a session vs more permanent storage places, but to each his/her own.

------------------------------------------------------------

Louis Landry

louis....@gmail.com

[Nicholas Dionysopoulos]

Hi Louis,

In a perfect world we'd have a semi-permanent, fast key/value storage (APC, Memcache, ....) at our disposal. But it's not a perfect world and the closest thing we have is the session and next thing we know we get to cram data to the session, despite not really wanting to. This leads to the session table being fragmented and, unless we regularly optimize it, becomes corrupt, killing our sites. But all the other alternatives either lead to the same issue (using our own temporary data tables) or come with severe performance and operational issues (using files). Essentially, we routinely get to consciously sacrifice architecture over compatibility with those imperfect hosts. Unfortunately, I don't see any other way, so it all comes down to implementing the best imperfect solution to the wrong problem for compatibility reasons.

Nicholas K. Dionysopoulos

Lead Developer, AkeebaBackup.com

Web: https://www.akeebabackup.com

Blog: http://www.dionysopoulos.me/blog

Sent from my iPad. Please excuse my brevity.

[Lobos]

People are saying that a session is not good to use to "reference" a carts contents to the "user", but there does not seem to be any other way... unless you are going to force people to register on your site to use your shopping cart which to me would seem to me foolish.

[Lobos]

sorry about my "foolish" grammer hahahhaa

[Sam Moffatt]

I'm about to jump on a plane but what you don't need a fast key/value
storage engine in this particular case because in reality what you're
doing is incredibly slow so latency in this case isn't an issue.

What you're doing is in fact "task" orientated not "session"
orientated in a sense. The task could be kicked off by a CLI
application without any issues. I had started on building a task and
taskset interface that would work well for generically storing a
series of related suspendable tasks (in your case your task set is a
backup with the individual task being scanning for directories via a
slow interface). I'll see if I can put it up on GitHub somewhere in a
branch to give you an idea of what I was thinking.

Cheers,

Sam Moffatt
http://pasamio.id.au

2011/9/4 Nicholas Dionysopoulos <niko...@gmail.com>:

[Nicholas Dionysopoulos]

Hi Sam,

Yes, please do. The interim soliton I have settled for since the last three and a half years is storing this kind of task data either on file or on a db record (the latter is a fallback because many hosts enjoy shutting down the MySQL connection after 1, 3 or 10 seconds after the initial connection leading to joyous "MySQL server has gone away" messages from PHP's MySQL and mysqli drivers, making it a might task making that work reliably). I would love to see if there is another solution, apart from implementing an additional adapter fora a key/value store like APC or Memcache.

Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com
Web: https://www.akeebabackup.com
Blog: http://www.dionysopoulos.me/blog

Sent from my iPad. Please excuse my brevity.

[Niels Braczek]

On 04.09.2011 21:33, Lobos wrote:

> People are saying that a session is not good to use to "reference" a carts
> contents to the "user", but there does not seem to be any other way...

Of course there is -- nobody keeps you from having a cart table i the
database using the session id as a reference, if the session storage
does not meet your requirements.

Regards,
Niels

--
| http://www.kolleg.de · Das Portal der Kollegs in Deutschland |
| http://www.bsds.de · BSDS Braczek Software- und DatenSysteme |
| Webdesign · Webhosting · e-Commerce · Joomla! Content Management |
------------------------------------------------------------------

[Louis Landry]

Or don't use the session id.  Create a cart_id field in auto-increment/serial integer form.  Then set a non-expiring cookie for that user with a value of the cart_id.  This really isn't an overly difficult problem to solve without having to use the session IMO.

The very requirements you have listed show that a user's session isn't the best place to store the information.  Seems like a case of square peg in a round hole to me.

------------------------------------------------------------

Louis Landry

louis....@gmail.com

[elin]

Wow great discussion and I'm thinking right now that for the table it should be MEDIUMTEXT for 1.7.1 if that's okay with all, I know it was LONGTEXT in CMS 1.5 but as has been pointed out that does seem excessive.  Let's say revisit the issue for CMS 3.0 but  ....

The real question that I was asking was ... if a session does hit the upper limit what should happen? Certainly not what happens now, which as Nic says is confusing/unhelpful to site users.  So shouldn't there be some way of handling it better? From what I can see that is part one of the framework issue (and why I posted here and not on CMS). 

Part 2 is handling this apparent issue with mysqli. Is there someone who would be willing to dig into that?

Part 3 is ... why can't people seem to use anything besides database? Or are these reports of not being able to do so some kind of user or developer error?  

Elin

[garyamort]

On Sunday, September 4, 2011 10:52:00 PM UTC-4, elin wrote:

Part 3 is ... why can't people seem to use anything besides database? Or are these reports of not being able to do so some kind of user or developer error?  

Perhaps libraries/joomla/application/application.php would shed some light on this:

Line 926:

//TODO: At some point we need to get away from having session data always in the db.

_createSession and checkSession store session information in the database regardless of what session storage engine is used.  Session /data/ is stored in the proper engine, but overall session stats are stored here.

session_id, client_id, guest, time, userid, and username are stored in the mysql database.

I've also run across some poorly behaving components that store data directly to the database instead of using the session handler - which causes all kinds of flakyness in trying to figure out why there is data in the session table.

 

[Lobos]

oks thanks for the good advice guys!

[Lobos]

Well I don't know about square peg or round hole, I just felt it was an easy solution to have the cart referenced from JSession - I liked how the cart reference carried over from guest user to logged in user and I kind of thought that if a user had "remember me" active it would imply that the session retains it's state and is not reset every x amount of minutes. Now I have to make my own key value table to do sort of the same thing - I am working in 1.5 so not sure if this is already available in new versions of J!

[Sam Moffatt]

Remember me just sets a long lived cookie and is used that to
authenticate the user. The session is wiped out when the session has
expired. Again, what you're doing is more task based not session
based. The convenience is jamming it into the session because its
there however it's not working for you because it isn't designed to do
what you're trying to do.

Cheers,

Sam Moffatt
http://pasamio.id.au

> To view this discussion on the web, visit

> https://groups.google.com/d/msg/joomla-dev-framework/-/7huOt6DE3eQJ.

[Lobos]

Oks, thanks Sam!