Re: SessionStore size - Bloat in sessionstore.js from Google and others causing major UI freezes

[Robert Kaiser]

Randell Jesup schrieb:
> e) Push the processing of the per-tab data over to a background
> thread/process.

That sounds like a good idea to me in any case. We should try to get as
much stuff away from the main thread as possible to ensure good
responsiveness of UI.

> P.S. It's nice to be back

\o/ welcome back \o/

Robert Kaiser

--
Note that any statements of mine - no matter how passionate - are never
meant to be offensive but very often as food for thought or possible
arguments that we as a community should think about. And most of the
time, I even appreciate irony and fun! :)

[Boris Zbarsky]

On 7/19/11 11:11 AM, Benjamin Smedberg wrote:
> because we don't normally treat saved sessions as "precious" data.

This is arguably a mistake....

I know I treat _my_ saved sessions as precious data.

-Boris

[Boris Zbarsky]

On 7/19/11 11:47 AM, Mike Shaver wrote:

> Across upgrades, though?

Across an upgrade, yes. Across downgrades, no, but that's because I
never downgrade. ;)

-Boris

[Randell Jesup]

I cross-posted this at first since .performance seemed pretty quiet
recently, but I think it makes sense for future comments to all be there
if people don't mind.

--
Randell Jesup, Mozilla Corporation
Remove ".news" for personal email

[Randell Jesup]

On 7/19/2011 11:46 AM, Mike Shaver wrote:
>> e) Push the processing of the per-tab data over to a background
>> thread/process.
>>

>> May play into work with Electrolysis. Not necessarily at odds with
>> other options above. Fetching data from each tab would be the
>> primary UI-blocker,
>
> IMO, that's not really OK long term (e10s-term) -- the tabs will need
> to report their state back to chrome, so that the blocking goes the
> other way.

Right - I didn't know the E10s architecture yet (on my list...) ;-)

>> Others? Any ideas out there?
>
> File-per-tab containing all the data for that tab, maybe? Or two
> files per tab, to avoid re-writing the big sessionstorage data every
> time, if it doesn't change that often. This would let us write out
> the current tab more frequently than background tabs, as well.

[ Hi Mike. Long time no chat :-) ]

That can be done with many of these ideas, especially the DB-based ones.
And file-per-tab is pretty much the same as a DB, it's just
filesystem-as-dumb-DB. :-) Not that it's necessarily bad.

Kyle (in .platform - that will teach me to cross-post) and Taras
suggested LevelDB over SQLite (we don't really need much in the way of
complexity here, mostly just safety against crashes and large blob
storage - which may speak to files, since filesystems are good at that).

An intern has expressed interest in trying some of the options proposed.
I dropped some dumb testcode in the bug to help.

[Joshua Cranmer]

On 7/19/2011 2:29 PM, Randell Jesup wrote:
> Kyle (in .platform - that will teach me to cross-post) and Taras
> suggested LevelDB over SQLite (we don't really need much in the way of
> complexity here, mostly just safety against crashes and large blob
> storage - which may speak to files, since filesystems are good at that).

LevelDB is designed to sensibly handle values from the small sizes to
multi-GB per value, or so I am told.

[Taras Glek]

On 07/19/2011 02:29 PM, Randell Jesup wrote:
> On 7/19/2011 11:46 AM, Mike Shaver wrote:
>>> e) Push the processing of the per-tab data over to a background
>>> thread/process.
>>>
>>> May play into work with Electrolysis. Not necessarily at odds with
>>> other options above. Fetching data from each tab would be the
>>> primary UI-blocker,
>>
>> IMO, that's not really OK long term (e10s-term) -- the tabs will need
>> to report their state back to chrome, so that the blocking goes the
>> other way.
>
> Right - I didn't know the E10s architecture yet (on my list...) ;-)
>
>>> Others? Any ideas out there?
>>
>> File-per-tab containing all the data for that tab, maybe? Or two
>> files per tab, to avoid re-writing the big sessionstorage data every
>> time, if it doesn't change that often. This would let us write out
>> the current tab more frequently than background tabs, as well.
>
> [ Hi Mike. Long time no chat :-) ]
>
> That can be done with many of these ideas, especially the DB-based ones.
> And file-per-tab is pretty much the same as a DB, it's just
> filesystem-as-dumb-DB. :-) Not that it's necessarily bad.

So just to clarify. Conceptually it may be the same as a db. In practice
keeping independent items in a file is a very bad idea. Ie if you have 7
tabs saved and then you close tab 3, you get in an ugly situation.
A database will try to pretend it's ok at expense of serious
fragmentation, requiring a vacuum, etc.

I think a sessionrestore directory with a json(or something else) per
tab is the only sane solution here. Let the filesystem do what it's good at.

The other benefit of file-per-tab is that they can be fsynced
independently without blocking io on other tab stores.

>
> Kyle (in .platform - that will teach me to cross-post) and Taras
> suggested LevelDB over SQLite (we don't really need much in the way of
> complexity here, mostly just safety against crashes and large blob
> storage - which may speak to files, since filesystems are good at that).
>
> An intern has expressed interest in trying some of the options proposed.
> I dropped some dumb testcode in the bug to help.

First step should be adding telemetry to record sizes of json files and
time to read/write them.

Taras
>