[RFC] Cleaning up sessionstore.js

[David Rajchenbach-Teller]

As many of you know, Session Restore is something of a performance hog,
for many reasons � we have reports of. One of the reasons is that we
store so very many things in sessionstore.js and sometimes keep stuff
for a very long time.

As part of bug 943352 & followup, we are considering automatically
cleanup some of the contents of sessionstore.js. Since people have
mentioned "webcompat" and "userdata loss" in the context of
sessionstore.js, I'd appreciate some feedback before we proceed.

So, here are a few things that I believe we could cleanup:
1. get rid of closed windows after a while;
2. get rid of closed tabs after a while;
3. get rid of old history entries of open tabs after a while;
4. get rid of POST data of history entries after a while;
5. get rid of DOM storage contents of open tabs after a while;
6. get rid of form data content of open tabs after a while;
...

Note that we don't have space usage number for each of these (bug 942340
should provide more insight).

If anybody feels that we are going to break one million websites (or one
million profiles), we would be interested to hear about this.

Cheers,
David

--
David Rajchenbach-Teller, PhD
Performance Team, Mozilla

[Till Schneidereit]

On Thu, Nov 28, 2013 at 11:56 AM, David Rajchenbach-Teller <
dte...@mozilla.com> wrote:

> As many of you know, Session Restore is something of a performance hog,

> for many reasons – we have reports of. One of the reasons is that we

> store so very many things in sessionstore.js and sometimes keep stuff
> for a very long time.
>
> As part of bug 943352 & followup, we are considering automatically
> cleanup some of the contents of sessionstore.js. Since people have
> mentioned "webcompat" and "userdata loss" in the context of
> sessionstore.js, I'd appreciate some feedback before we proceed.
>
> So, here are a few things that I believe we could cleanup:
> 1. get rid of closed windows after a while;
> 2. get rid of closed tabs after a while;
> 3. get rid of old history entries of open tabs after a while;
> 4. get rid of POST data of history entries after a while;
> 5. get rid of DOM storage contents of open tabs after a while;
> 6. get rid of form data content of open tabs after a while;
>

This would all be tackled after we did other things like getting rid of all
history entries for iframes, which won't be restored in any case, right?

I'm asking because all of these seem problematic to me, to be honest. The
first two the least - I think it's mostly ok to have to go through the
history itself to look up URLs of windows/tabs that were closed in a
previous session. But for the others, I'd very much expect issues. Right
now, our session storage is wonderfully robust (as long as no bugs are
encountered) in that I can do things like starting to fill in forms without
having to worry about the browser crashing or me forgetting to actually
send the form before restarting the browser.

As a concrete example, I start writing my Status Board[1] entries during
the week, and only send them off on Mondays. If we were to get rid of form
data after some period of time, I couldn't do this anymore. Even worse: I
might not know when exactly form data is discarded, so it'd *seem* to work
just fine for a while, and I might invest quite some time in writing my
update, only to lose it all of a sudden because Firefox decided that I
don't need this data anymore.

Similar scenarios can probably be thought up/occur for 4 and 5, too.

As for 3, we could maybe gradually get rid of entries, oldest first. So it
wouldn't be a hard cut-off, but a gradual loss of entries which are less
and less likely to be of interest, anyway.


[1]: http://benjamin.smedbergs.us/weekly-updates.fcgi/

> ...
>
> Note that we don't have space usage number for each of these (bug 942340
> should provide more insight).
>
> If anybody feels that we are going to break one million websites (or one
> million profiles), we would be interested to hear about this.
>
> Cheers,
> David
>
> --
> David Rajchenbach-Teller, PhD
> Performance Team, Mozilla

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>

[David Rajchenbach-Teller]

On 11/28/13 1:33 PM, Till Schneidereit wrote:
> This would all be tackled after we did other things like getting rid
> of all history entries for iframes, which won't be restored in any
> case, right?

We're not sure about the relative priorities of this cleanup vs.
removing the history entries for dynamic iframes that cannot be
restored. That will probably depend on the results of bug 942340.

> As a concrete example, I start writing my Status Board[1] entries during
> the week, and only send them off on Mondays. If we were to get rid of
> form data after some period of time, I couldn't do this anymore. Even
> worse: I might not know when exactly form data is discarded, so it'd
> *seem* to work just fine for a while, and I might invest quite some time
> in writing my update, only to lose it all of a sudden because Firefox
> decided that I don't need this data anymore.

Good point. If we head in this direction, we definitely need to mark
that data (form, POST or DOM storage) as unneeded only if the tab hasn't
been active at all during the interval. Also, I was thinking of a time
to live of at least 1 week, but I'm willing to make it 1 month. The key
idea is to make sure that data eventually disappears, rather than
staying forever.

> Similar scenarios can probably be thought up/occur for 4 and 5, too.
>
> As for 3, we could maybe gradually get rid of entries, oldest first. So
> it wouldn't be a hard cut-off, but a gradual loss of entries which are
> less and less likely to be of interest, anyway.

That was the idea, yes.

[Till Schneidereit]

On Thu, Nov 28, 2013 at 1:55 PM, David Rajchenbach-Teller <
dte...@mozilla.com> wrote:

> On 11/28/13 1:33 PM, Till Schneidereit wrote:
> > This would all be tackled after we did other things like getting rid
> > of all history entries for iframes, which won't be restored in any
> > case, right?
>
> We're not sure about the relative priorities of this cleanup vs.
> removing the history entries for dynamic iframes that cannot be
> restored. That will probably depend on the results of bug 942340.
>

Except if it's *much* more effort to reduce the amount of stored data
that's simply useless, it seems to me that it's strictly preferable to do
that regardless of the telemetry results.

>
> > As a concrete example, I start writing my Status Board[1] entries during
> > the week, and only send them off on Mondays. If we were to get rid of
> > form data after some period of time, I couldn't do this anymore. Even
> > worse: I might not know when exactly form data is discarded, so it'd
> > *seem* to work just fine for a while, and I might invest quite some time
> > in writing my update, only to lose it all of a sudden because Firefox
> > decided that I don't need this data anymore.
>
> Good point. If we head in this direction, we definitely need to mark
> that data (form, POST or DOM storage) as unneeded only if the tab hasn't
> been active at all during the interval. Also, I was thinking of a time
> to live of at least 1 week, but I'm willing to make it 1 month. The key
> idea is to make sure that data eventually disappears, rather than
> staying forever.
>

Maybe the size of that data could also be a factor? Removing one or a few
short strings will probably not save much data, whereas removing, say, a
large image from POST data most certainly will.

>
> > Similar scenarios can probably be thought up/occur for 4 and 5, too.
> >
> > As for 3, we could maybe gradually get rid of entries, oldest first. So
> > it wouldn't be a hard cut-off, but a gradual loss of entries which are
> > less and less likely to be of interest, anyway.
>
> That was the idea, yes.
>

Cool.

[Honza Bambas]

On 11/28/2013 12:56 PM, David Rajchenbach-Teller wrote:
> As many of you know, Session Restore is something of a performance hog,

> for many reasons � we have reports of. One of the reasons is that we

> store so very many things in sessionstore.js and sometimes keep stuff
> for a very long time.
>
> As part of bug 943352 & followup, we are considering automatically
> cleanup some of the contents of sessionstore.js. Since people have
> mentioned "webcompat" and "userdata loss" in the context of
> sessionstore.js, I'd appreciate some feedback before we proceed.
>
> So, here are a few things that I believe we could cleanup:
> 1. get rid of closed windows after a while;

Definitely, also from a privacy POV.

> 2. get rid of closed tabs after a while;

Ditto

> 3. get rid of old history entries of open tabs after a while;
> 4. get rid of POST data of history entries after a while;
> 5. get rid of DOM storage contents of open tabs after a while;
> 6. get rid of form data content of open tabs after a while;

4-6: from a user POV - I have to agree also with Till, I really like
that form data are not lost *for ages*, I always write a bug report and
then leave it unsent to collect more data first, as well as any bug
comments. And I'm really enjoying how well Fx keeps these data
unbroken. That should stay.

[Matthew N.]

On 11/28/13, 7:15 AM, Honza Bambas wrote:
> On 11/28/2013 12:56 PM, David Rajchenbach-Teller wrote:
>> As many of you know, Session Restore is something of a performance hog,
>> for many reasons � we have reports of. One of the reasons is that we
>> store so very many things in sessionstore.js and sometimes keep stuff
>> for a very long time.

Do we know that these issues affect a large number of users and not just
tab hoarders like myself?

>> So, here are a few things that I believe we could cleanup:
>> 1. get rid of closed windows after a while;
> Definitely, also from a privacy POV.
>> 2. get rid of closed tabs after a while;
> Ditto
>> 3. get rid of old history entries of open tabs after a while;
>> 4. get rid of POST data of history entries after a while;
>> 5. get rid of DOM storage contents of open tabs after a while;
>> 6. get rid of form data content of open tabs after a while;
> 4-6: from a user POV - I have to agree also with Till, I really like
> that form data are not lost *for ages*, I always write a bug report and
> then leave it unsent to collect more data first, as well as any bug
> comments. And I'm really enjoying how well Fx keeps these data
> unbroken. That should stay.

I agree that #1 and #2 should expire but I don't think expiring 3-6 is
worth the loss of functionality and trust in the browser. Fixing #1 and
#2 would also probably reduce the need for 3-6 since the browser default
is to open the homepage on every launch instead of restoring the session
and we know that most users don't change defaults.

It's not uncommon for me to go back to a tab group for a side project
that I haven't worked on for a few months and want to resume where I
left off. #3, #4, and #6 would definitely hinder that. It's hard to say
if #5 would without knowing how websites are using DOM storage. To be
clear, is that just referring to window.sessionStorage?

Cheers,
Matthew N. [:MattN]

[David Rajchenbach-Teller]

On 11/29/13 12:15 AM, Matthew N. wrote:
> On 11/28/13, 7:15 AM, Honza Bambas wrote:
>> On 11/28/2013 12:56 PM, David Rajchenbach-Teller wrote:
>>> As many of you know, Session Restore is something of a performance hog,
>>> for many reasons � we have reports of. One of the reasons is that we
>>> store so very many things in sessionstore.js and sometimes keep stuff
>>> for a very long time.
>
> Do we know that these issues affect a large number of users and not just
> tab hoarders like myself?

We have reports of single tabs taking 10+Mb (bug 942601, bug 934935).
We're in the process of adding telemetry to find out how widespread this
kind of situation is and acquire more visibility (bug 942340).

> It's not uncommon for me to go back to a tab group for a side project
> that I haven't worked on for a few months and want to resume where I
> left off. #3, #4, and #6 would definitely hinder that. It's hard to say
> if #5 would without knowing how websites are using DOM storage. To be
> clear, is that just referring to window.sessionStorage?

It is.

[Robert Kaiser]

David Rajchenbach-Teller schrieb:

> As part of bug 943352 & followup, we are considering automatically
> cleanup some of the contents of sessionstore.js.

Just for my understanding (I have commented to users with huge, e.g.
~100MB sessionstore.js in bugs as well), I thought we were working on a
rewrite of session store anyhow that would not kepp info of all tabs in
one file?
I think I have heard that we'd need to do this because of e10s anyhow at
some point, and that it also would help our startup if we didn't need to
load all session store data for all tabs at once but could do it per tab
when they are actually restored/loaded.

Now I fully agree with trying to not store things we probably should not
keep anyhow, I just wonder if it might make sense to take into account
where session store is going.

Of course, I might be completely mistaken with what I thought I heard
there, so please correct me if I'm wrong.

KaiRo

[David Rajchenbach-Teller]

On 11/29/13 4:47 AM, Robert Kaiser wrote:
> Just for my understanding (I have commented to users with huge, e.g.
> ~100MB sessionstore.js in bugs as well), I thought we were working on a
> rewrite of session store anyhow that would not kepp info of all tabs in
> one file?
>
> I think I have heard that we'd need to do this because of e10s anyhow at
> some point, and that it also would help our startup if we didn't need to
> load all session store data for all tabs at once but could do it per tab
> when they are actually restored/loaded.

This redesign is bound to happen, sooner or later, although we are not
actively working on this at the moment. It was blocked in part by the
fact that the implementation of Session Restore had become a mess
(that's now mostly solved) and by the fact that it was completely
monolithic (also solved in many cases). I hope that we can now resume
that work on Q1, but that's not a promise.

Note tat this is actually not required by e10s, but if done well, this
should indeed improve startup, as well as memory use.

> Now I fully agree with trying to not store things we probably should not
> keep anyhow, I just wonder if it might make sense to take into account
> where session store is going.

Well, any change that would split sessionstore.js is going to need even
more effort in ensuring that we collect the garbage, so I believe that
this is a useful first step.

[Wayne]

I have similar workflow and would also consider 3-6 to be dataloss.

[Dao]

On 28.11.2013 16:15, Honza Bambas wrote:
> On 11/28/2013 12:56 PM, David Rajchenbach-Teller wrote:
>> As many of you know, Session Restore is something of a performance hog,
>> for many reasons � we have reports of. One of the reasons is that we
>> store so very many things in sessionstore.js and sometimes keep stuff
>> for a very long time.
>>
>> As part of bug 943352 & followup, we are considering automatically
>> cleanup some of the contents of sessionstore.js. Since people have
>> mentioned "webcompat" and "userdata loss" in the context of
>> sessionstore.js, I'd appreciate some feedback before we proceed.
>>
>> So, here are a few things that I believe we could cleanup:
>> 1. get rid of closed windows after a while;
> Definitely, also from a privacy POV.

The closed tabs' and windows' URLs and visit date and time are easily
accessible from your browsing history. I don't see how arbitrarily
removing them from the closed tabs/windows lists would improve privacy.

[kalle.pih...@gmail.com]

torstai 28. marraskuuta 2013 13.56.13 UTC+2 David Rajchenbach-Teller kirjoitti:
> As many of you know, Session Restore is something of a performance hog,

> for many reasons - we have reports of. One of the reasons is that we

I have been wrestling with a big sessionstore.js since I discovered tab-groups, I have been trying to find remedies to performance and recovery and have not found anything. A few times per year I go looking for inspiration and it seems there is little information about the status of these upgrades.

Is there some way to do steps 1 to 6 manually (perhaps a plug-in)?

At many times at the end of a day I like to think that I am at a place that I could do a clean-up.

Mostly my tabs are used for side projects and tool groups so they do not need historical context, I would just like the URL stored. I would very much like to have the start-up faster, tag group switching faster and less clutter in the sessionstore if it were to crash and need recovery.

Leaving more historical data by default could be done in pinned tabs perhaps (though not needed by me) and this would be a way to preserve state in those tabs that are important to maintain forms data etc.

From a privacy point of view this is going to become increasingly important. When the session store breaks it often leaves old copies that remain unless manually removed. It would be nice if they contained a bit less old data.

KalleP