Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

A uniform approach to discarding re-generable stuff

127 views
Skip to first unread message

Nicholas Nethercote

unread,
Aug 30, 2011, 10:26:32 PM8/30/11
to dev-pl...@lists.mozilla.org
In today's MemShrink meeting we briefly discussed
https://bugzilla.mozilla.org/show_bug.cgi?id=675539, which suggests
unloading tabs that haven't been used in a long time so conserve
memory. We concluded that this sounded like (to use a Gal-ism) a
trail of tears due to data loss concerns.

However, there is clearly lots of regenerable content in the browser,
and we have a number of mechanisms for discarding it. Some things we
discard based on timers, and some things we discard based on memory
pressure events, and there are probably other triggers as well. Some
examples I'm aware of:

- Decoded images are dropped in the background tab after 10 to 20
seconds. https://bugzilla.mozilla.org/show_bug.cgi?id=668634 is open
to do likewise for foreground tabs on memory pressure events.
(There's also https://bugzilla.mozilla.org/show_bug.cgi?id=573583 and
https://bugzilla.mozilla.org/show_bug.cgi?id=661304, but they're about
decoding fewer images to begin with, which is a different strategy,)

- Method JIT code is discarded gradually, based on some kind of
time-based exponential decay mechanism.
https://bugzilla.mozilla.org/show_bug.cgi?id=681815 is open to also
discard JIT code (both method JIT and trace JIT? unclear) on memory
pressure events.

- JavaScript GC can arguably be viewed under this lens. It has
multiple triggers, based on allocations and memory pressure events.

- There's presumably a lot more: bfcache, SQLite caches, etc, etc.

Memory pressure events are triggered in the following circumstances:

- On Android, when Fennec goes into the background
(https://bugzilla.mozilla.org/show_bug.cgi?id=618537).

- From about:memory, when the "Minimize memory usage" button is pressed.

- https://bugzilla.mozilla.org/show_bug.cgi?id=664291 is open to
trigger memory pressure events when certain memory statistics
(virtual/physical memory available, hard page fault count) reach
certain thresholds.

- Any others?

Anyway, it seems like this discarding of re-generable data is very ad
hoc and could do with some more uniformity. Fennec stands to benefit
particularly from this.
Maybe it's just a matter of ensuring that the memory pressure events
are wired up to all the appropriate places? In which case maybe we
just need a tracking bug for that.

Or maybe it makes sense to do something more than that. Could we have
memory pressure notifications for individual tabs, which can be
triggered when the tab is backgrounded?

I'd be interested to hear any suggestions, and also to hear about
other examples of regenerable data that I haven't mentioned above, and
their discard triggers. Thanks!

Nick

Boris Zbarsky

unread,
Aug 30, 2011, 10:43:48 PM8/30/11
to
On 8/30/11 10:26 PM, Nicholas Nethercote wrote:
> - There's presumably a lot more: bfcache, SQLite caches, etc, etc.

Anything using nsExpirationTracker. That includes the image cache,
textrun cache, some sort of font data, looks like some sort of layer
data for painting, bfcache stuff of some sort, I think, and the canvas
image cache.

There's various style data (rule cascades, computed structs, etc) that's
lazily generated and could in theory be nuked if needed. Right now it
just hangs around once computed and does not get expired.

SVG attribute style rules are lazily allocated and could be regenerated
as needed.

If we move enough state out of the frame tree (a definite goal we're
working on), the entire frame tree (and hence style context tree and
rulenode tree) can become discardable. This is a ways off yet.

I wonder whether we could discard canvas backing stores when seriously
in need of memory by encoding to png? The problem is that this involves
allocating memory for the png before freeing the canvas backing store....

The rulenode tree is not discardable per se but deallocation of
rulenodes happens via a mark-and-sweep GC mechanism triggered by the
"number of style contexts destroyed since last ruletree gc" counter
reaching a certain threshold. We could probably do such a GC pass on
memory pressure, but rulenodes come out of the presshell arena anyway,
so we wouldn't be releasing memory back to the OS. :(

There are various arena and recycling allocators that use freelists that
could probably be made to play nicer with memory pressure notifications
somehow.

> Anyway, it seems like this discarding of re-generable data is very ad
> hoc and could do with some more uniformity.

We have two kinds of discarding. One is "discard as much as you can
right now" and one is "this is cached data that's likely to need to be
used lots of times in a row, but is probably safe to drop if it hasn't
been used in a while".

Memory pressure notifications are supposed to handle the former, while
expiration trackers are supposed to handle the latter.

In practice, we don't listen to memory pressure enough, as you point
out; we should fix that no matter what.

> Or maybe it makes sense to do something more than that. Could we have
> memory pressure notifications for individual tabs, which can be
> triggered when the tab is backgrounded?

A number of these caches are process-global at the moment. Of course
once we have separate tab processes it gets easier to send
memory-pressure to some subset of them...

-Boris

Robert O'Callahan

unread,
Aug 30, 2011, 11:24:18 PM8/30/11
to Boris Zbarsky, dev-pl...@lists.mozilla.org
On Wed, Aug 31, 2011 at 2:43 PM, Boris Zbarsky <bzba...@mit.edu> wrote:

> If we move enough state out of the frame tree (a definite goal we're
> working on), the entire frame tree (and hence style context tree and
> rulenode tree) can become discardable. This is a ways off yet.
>

How far off is it? Bug 90268 is the biggest remaining issue I know of and
that's very close to done. What else is there? Throwing out the frame trees
(and style data!) for old background tabs could work quite well.

In practice, we don't listen to memory pressure enough, as you point out; we
> should fix that no matter what.


Probably we should make all nsExpirationTrackers listen for memory pressure
automatically.


> Or maybe it makes sense to do something more than that. Could we have
>> memory pressure notifications for individual tabs, which can be
>> triggered when the tab is backgrounded?
>>
>
> A number of these caches are process-global at the moment. Of course once
> we have separate tab processes it gets easier to send memory-pressure to
> some subset of them...
>

I dunno if multiple content processes will help much. We probably need to
route memory pressure notifications to old background tabs explicitly.

Rob
--
"If we claim to be without sin, we deceive ourselves and the truth is not in
us. If we confess our sins, he is faithful and just and will forgive us our
sins and purify us from all unrighteousness. If we claim we have not sinned,
we make him out to be a liar and his word is not in us." [1 John 1:8-10]

Robert O'Callahan

unread,
Aug 30, 2011, 11:25:17 PM8/30/11
to Boris Zbarsky, dev-pl...@lists.mozilla.org
On Wed, Aug 31, 2011 at 3:24 PM, Robert O'Callahan <rob...@ocallahan.org>wrote:

> On Wed, Aug 31, 2011 at 2:43 PM, Boris Zbarsky <bzba...@mit.edu> wrote:
>

>> If we move enough state out of the frame tree (a definite goal we're
>> working on), the entire frame tree (and hence style context tree and
>> rulenode tree) can become discardable. This is a ways off yet.
>>
>

> How far off is it? Bug 90268 is the biggest remaining issue I know of and
> that's very close to done. What else is there? Throwing out the frame trees
> (and style data!) for old background tabs could work quite well.
>

I guess we'd want to limit it to documents that aren't getting forced
flushes due to script activity.

Boris Zbarsky

unread,
Aug 30, 2011, 11:37:18 PM8/30/11
to
On 8/30/11 11:24 PM, Robert O'Callahan wrote:
> On Wed, Aug 31, 2011 at 2:43 PM, Boris Zbarsky<bzba...@mit.edu> wrote:
>
>> If we move enough state out of the frame tree (a definite goal we're
>> working on), the entire frame tree (and hence style context tree and
>> rulenode tree) can become discardable. This is a ways off yet.
>>
>
> How far off is it? Bug 90268 is the biggest remaining issue I know of and
> that's very close to done. What else is there?

Scroll positions and maybe selections are the main things that come to mind.

I think at this point we can save caret positions on the content, right?

> Throwing out the frame trees (and style data!) for old background tabs could work quite well.

If we could throw out the whole presshell arena in the process, yeah.
If not, then it doesn't help much, right?

> Probably we should make all nsExpirationTrackers listen for memory pressure
> automatically.

Yeah. Wish that were doable without xpcom goop. :(

>> A number of these caches are process-global at the moment. Of course once
>> we have separate tab processes it gets easier to send memory-pressure to
>> some subset of them...
>
> I dunno if multiple content processes will help much. We probably need to
> route memory pressure notifications to old background tabs explicitly.

That helps with per-tab data structures like frame trees, but not things
like image and textrun caches that are process-global.

-Boris

Robert O'Callahan

unread,
Aug 31, 2011, 12:06:07 AM8/31/11
to Boris Zbarsky, dev-pl...@lists.mozilla.org
On Wed, Aug 31, 2011 at 3:37 PM, Boris Zbarsky <bzba...@mit.edu> wrote:

> On 8/30/11 11:24 PM, Robert O'Callahan wrote:
>
>> On Wed, Aug 31, 2011 at 2:43 PM, Boris Zbarsky<bzba...@mit.edu> wrote:
>>
>> If we move enough state out of the frame tree (a definite goal we're
>>> working on), the entire frame tree (and hence style context tree and
>>> rulenode tree) can become discardable. This is a ways off yet.
>>>
>>>
>> How far off is it? Bug 90268 is the biggest remaining issue I know of and
>> that's very close to done. What else is there?
>>
>
> Scroll positions


Already saved and restored using nsPresState.


> and maybe selections are the main things that come to mind.
>

Bug 619273 moves selection state bits to content; Mats is on it. There may
not be any other issues with selection.

I think at this point we can save caret positions on the content, right?


Yes.


>
> Throwing out the frame trees (and style data!) for old background tabs
>> could work quite well.
>>
>
> If we could throw out the whole presshell arena in the process, yeah. If
> not, then it doesn't help much, right?


Well, let's do that then :-).


> A number of these caches are process-global at the moment. Of course once
>>> we have separate tab processes it gets easier to send memory-pressure to
>>> some subset of them...
>>>
>>
>> I dunno if multiple content processes will help much. We probably need to
>> route memory pressure notifications to old background tabs explicitly.
>>
>
> That helps with per-tab data structures like frame trees, but not things
> like image and textrun caches that are process-global.


Yeah, it's kind of a pain to deal with objects shared across documents.

Boris Zbarsky

unread,
Aug 31, 2011, 12:16:41 AM8/31/11
to
On 8/31/11 12:06 AM, Robert O'Callahan wrote:
>> Scroll positions
>
> Already saved and restored using nsPresState.

That's the theory, but does it actually work in practice? Last I had
checked, it didn't quite (though I can't recall whether the problem was
for the root scrollframe or for non-root scrollframes).

>> and maybe selections are the main things that come to mind.
>
> Bug 619273 moves selection state bits to content; Mats is on it. There may
> not be any other issues with selection.

Right. We're working on this stuff; we're just not quite done with it yet.

>> If we could throw out the whole presshell arena in the process, yeah. If
>> not, then it doesn't help much, right?
>
> Well, let's do that then :-).

Sure; just have to be careful with style contexts.

-Boris

Boris Zbarsky

unread,
Aug 31, 2011, 2:21:45 AM8/31/11
to
On 8/30/11 11:25 PM, Robert O'Callahan wrote:
> I guess we'd want to limit it to documents that aren't getting forced
> flushes due to script activity.

We could do that with an expiration tracker, I suspect. We could still
get bad behavior if the script timeout is just exactly longer than our
expiration time, of course...

-Boris

Neil

unread,
Aug 31, 2011, 4:38:00 AM8/31/11
to
Boris Zbarsky wrote:

> If we move enough state out of the frame tree (a definite goal we're
> working on), the entire frame tree (and hence style context tree and
> rulenode tree) can become discardable.

(How?) Would this affect tabs such as about:addons or were you just
considering HTML documents?

--
Warning: May contain traces of nuts.

Ehsan Akhgari

unread,
Aug 31, 2011, 9:24:50 AM8/31/11
to Neil, dev-pl...@lists.mozilla.org
On 11-08-31 4:38 AM, Neil wrote:
> Boris Zbarsky wrote:
>
>> If we move enough state out of the frame tree (a definite goal we're
>> working on), the entire frame tree (and hence style context tree and
>> rulenode tree) can become discardable.
>
> (How?) Would this affect tabs such as about:addons or were you just
> considering HTML documents?

I think we can just consider HTML documents.

Ehsan

Benjamin Smedberg

unread,
Aug 31, 2011, 11:11:50 AM8/31/11
to Nicholas Nethercote, dev-pl...@lists.mozilla.org
On 8/30/2011 10:26 PM, Nicholas Nethercote wrote:
> In today's MemShrink meeting we briefly discussed
> https://bugzilla.mozilla.org/show_bug.cgi?id=675539, which suggests
> unloading tabs that haven't been used in a long time so conserve
> memory. We concluded that this sounded like (to use a Gal-ism) a
> trail of tears due to data loss concerns.
I've never really understood why it's quite so hard to save the state of
an entire webpage, discounting for the moment plugins. Can anyone
explain why we can't use some sort of "super-XDR" to serialize an entire
DOM+JS tree to disk? This would help us a lot in a bunch of scenarios,
including silent restarts that could make session restore a lot less
datalossy. I think it might also make it possible to really support
users who want to keep their 100s-of-tabs workflow without making us hog
huge amounts of memory.

Although I fully support the intermediate solutions also proposed such
as tearing down the frame tree and discarding other data.

--BDS

Ehsan Akhgari

unread,
Aug 31, 2011, 11:19:30 AM8/31/11
to Benjamin Smedberg, dev-pl...@lists.mozilla.org, Nicholas Nethercote
On 11-08-31 11:11 AM, Benjamin Smedberg wrote:
> On 8/30/2011 10:26 PM, Nicholas Nethercote wrote:
>> In today's MemShrink meeting we briefly discussed
>> https://bugzilla.mozilla.org/show_bug.cgi?id=675539, which suggests
>> unloading tabs that haven't been used in a long time so conserve
>> memory. We concluded that this sounded like (to use a Gal-ism) a
>> trail of tears due to data loss concerns.
> I've never really understood why it's quite so hard to save the state of
> an entire webpage, discounting for the moment plugins. Can anyone
> explain why we can't use some sort of "super-XDR" to serialize an entire
> DOM+JS tree to disk?

I can't speak about the JS engine, but on the content side, I think it's
just a lot of work that needs to be done to make sure that everything in
the content tree, and the stuff that are held by the content tree, are
serializable. I don't think there are any real technical challenges to
make having this impossible.

Cheers,
Ehsan

Boris Zbarsky

unread,
Aug 31, 2011, 12:17:19 PM8/31/11
to
On 8/31/11 11:11 AM, Benjamin Smedberg wrote:
> I've never really understood why it's quite so hard to save the state of
> an entire webpage

It's not, probably except for the fact that the code needs to be
written. And that serializing cyclic object graphs is a bit of a pain.

> Can anyone explain why we can't use some sort of "super-XDR" to serialize an entire
> DOM+JS tree to disk?

We probably can, with enough work.

But note that any time the data structure changes things like our
current fastload and XDR can't be used to migrate across the change. So
we'd still have a problem across updates. :(

-Boris

Boris Zbarsky

unread,
Aug 31, 2011, 12:17:55 PM8/31/11
to
On 8/31/11 4:38 AM, Neil wrote:
> Boris Zbarsky wrote:
>
>> If we move enough state out of the frame tree (a definite goal we're
>> working on), the entire frame tree (and hence style context tree and
>> rulenode tree) can become discardable.
>
> (How?) Would this affect tabs such as about:addons or were you just
> considering HTML documents?

I was assuming we'd do this for everything, but it may be worth sticking
to HTML only at first if there's state stored in XUL frames (which there
probably is).

-Boris


Mike Shaver

unread,
Aug 31, 2011, 12:32:59 PM8/31/11
to Nicholas Nethercote, dev-pl...@lists.mozilla.org
On Tue, Aug 30, 2011 at 10:26 PM, Nicholas Nethercote
<n.neth...@gmail.com> wrote:
> Anyway, it seems like this discarding of re-generable data is very ad
> hoc and could do with some more uniformity.  Fennec stands to benefit
> particularly from this.

Fennec on Android could also use ashmem for some things (I've
suggested the image and network caches in the past, maybe all the
canvas/surface memory if we can do that), which would let the OS
discard it for us under memory pressure.

http://elinux.org/Android_Kernel_Features#ashmem

AFAICT the basic model is that before you use something you pin it
based on the handle. If the pin fails, then you know that the kernel
dropped it, so you reallocate and regenerate. Can mostly just be
handled like a cache miss, I suspect, and similar to what we would do
on desktop around the regeneration.

We probably (heh) don't want to run a syscall every time we check for
JIT code, but we could use a decaying timer to keep a compartment's
JIT code space pinned for a while (5 mins?). Might also help with
fragmentation in those cases, in fact!

Mike

Mike Shaver

unread,
Aug 31, 2011, 12:37:34 PM8/31/11
to Boris Zbarsky, dev-pl...@lists.mozilla.org
On Wed, Aug 31, 2011 at 12:17 PM, Boris Zbarsky <bzba...@mit.edu> wrote:
> On 8/31/11 11:11 AM, Benjamin Smedberg wrote:
>>
>> I've never really understood why it's quite so hard to save the state of
>> an entire webpage
>
> It's not, probably except for the fact that the code needs to be written.
>  And that serializing cyclic object graphs is a bit of a pain.

There are some parts that are tricky in our internal state, and I
don't want to have to write the XDR logic this time around, but it
might be feasible.

Things we'd need to figure out:
- chrome and extension state that's tied to object/page/window
identity, such as via weak maps

- how to reload without being incredibly slow due to I/O -- we have
found a few times that loading from the network can be faster than
reading from the disk cache

- yeah, plugins

- websocket and long-poll

- video content (can't really reload it, necessarily)

The rest is all just a simple matter of programming, and a lot of it:
serialize all the XPConnect state, wrapper relationships, DOM
wrappers, blah blah. It actually sounds like it would be fairly fun,
especially if there's a meaningful and isolated chunk to start with
(maybe the frame tree?).

Mike

Neil

unread,
Aug 31, 2011, 12:42:22 PM8/31/11
to
Boris Zbarsky wrote:

> If we move enough state out of the frame tree (a definite goal we're
> working on), the entire frame tree (and hence style context tree and
> rulenode tree) can become discardable. This is a ways off yet.

How does this compare to bfcache? (SeaMonkey currently uses bfcache to
"discard" a tab, but it would be nice if there was an API to do this.)

Boris Zbarsky

unread,
Aug 31, 2011, 1:44:00 PM8/31/11
to
On 8/31/11 12:42 PM, Neil wrote:
> Boris Zbarsky wrote:
>
>> If we move enough state out of the frame tree (a definite goal we're
>> working on), the entire frame tree (and hence style context tree and
>> rulenode tree) can become discardable. This is a ways off yet.
>
> How does this compare to bfcache?

bfache saves an in-memory representation of the DOM and the frame tree.
In terms of memory behavior basically no different from just hiding
the old tab without unloading it and giving the user a new tab for their
new pageload. The one difference is that bfcache stops plug-in instances.

-Boris

Justin Dolske

unread,
Aug 31, 2011, 5:35:08 PM8/31/11
to

Should we consider eventually jettisoning bfcache in favor of a single
mechanism built on this frame discarding stuff? I've never touched it,
but my impression is that bfcache has long been a source of pain?

Justin

Mike Shaver

unread,
Aug 31, 2011, 5:44:26 PM8/31/11
to Justin Dolske, dev-pl...@lists.mozilla.org

I could see rebuilding bfcache on top of the frame-regeneration stuff,
but I think we would need to preserve DOM and JS state for the feature
to work well. Maybe plugins, I don't know if that actually matters in
a lot of cases.

(I think that the motivation -- dominant action is back/forward -- has
changed a bit in significance with the advent of tabbed browsing, but
at least one-step-back is still pretty important in my browsing, I've
found.)

Mike

Boris Zbarsky

unread,
Aug 31, 2011, 6:02:38 PM8/31/11
to
On 8/31/11 5:35 PM, Justin Dolske wrote:
> Should we consider eventually jettisoning bfcache in favor of a single
> mechanism built on this frame discarding stuff? I've never touched it,
> but my impression is that bfcache has long been a source of pain?

The use cases are somewhat different.

The point of bfcache is to give you very fast going back in history.
That's why it keeps everything in memory, etc. Any sort of "read in a
few tens of MB of data from disk" kind of operation will be slower than
a bfcache restore; possibly slower than an actual page load.

We could nuke the frame tree in bfcache, I suppose, but we'd still want
to preserve the DOM and the like, no?

-Boris

Robert O'Callahan

unread,
Aug 31, 2011, 8:16:01 PM8/31/11
to Boris Zbarsky, dev-pl...@lists.mozilla.org


I discussed this with bryner back in the day and back then he wanted to keep
the frame tree around for faster restores (even if we could have regenerated
it, which we were nowhere near able to back then).

Robert O'Callahan

unread,
Aug 31, 2011, 8:16:58 PM8/31/11
to Justin Dolske, dev-pl...@lists.mozilla.org
On Thu, Sep 1, 2011 at 9:35 AM, Justin Dolske <dol...@mozilla.com> wrote:

> Should we consider eventually jettisoning bfcache in favor of a single
> mechanism built on this frame discarding stuff? I've never touched it, but
> my impression is that bfcache has long been a source of pain?
>

My impression is that the pain comes largely from docshell and session
history management issues which are mostly orthogonal to what we're
discussing here.

Robert O'Callahan

unread,
Aug 31, 2011, 8:29:39 PM8/31/11
to Mike Shaver, Boris Zbarsky, dev-pl...@lists.mozilla.org
On Thu, Sep 1, 2011 at 4:37 AM, Mike Shaver <mike....@gmail.com> wrote:

> Things we'd need to figure out:
> - chrome and extension state that's tied to object/page/window
> identity, such as via weak maps
>
> - how to reload without being incredibly slow due to I/O -- we have
> found a few times that loading from the network can be faster than
> reading from the disk cache
>
> - yeah, plugins
>
> - websocket and long-poll
>
> - video content (can't really reload it, necessarily)
>

Would probably work fine in practice just saving the current position and
other element state and seeking to that position on restore.

In general, though, handling in-progress loads is going to be tough. I
suppose you can always give up and fail them.

WebGL would be interesting. If you can't get all the state out of the native
context, you'll need to mirror some of it, which could cost performance. 2D
canvas is also an issue but Azure's statelessness makes it a lot easier.

The rest is all just a simple matter of programming, and a lot of it:
> serialize all the XPConnect state, wrapper relationships, DOM
> wrappers, blah blah. It actually sounds like it would be fairly fun,
> especially if there's a meaningful and isolated chunk to start with
> (maybe the frame tree?).
>

I think for anything that we support regenerating from local data, like the
frame tree (soon), it wouldn't be worth serializing/deserializing --- we'd
just discard it and regenerate it. Less code to write, and probably faster
anyway.

Full serialization of Web page state sounds quite awesome. Doing it across
browser updates would be even more awesome --- invisible security updates,
woohoo! It'd be fun to have browser features appearing during a page
session, and probably wouldn't even cause a lot of problems in practice.
Unfortunately it'd be a massive and hugely invasive project.

Cedric Vivier

unread,
Sep 1, 2011, 12:55:59 AM9/1/11
to rob...@ocallahan.org, Boris Zbarsky, dev-pl...@lists.mozilla.org, Mike Shaver
On Thu, Sep 1, 2011 at 08:29, Robert O'Callahan <rob...@ocallahan.org> wrote:
> WebGL would be interesting. If you can't get all the state out of the native
> context, you'll need to mirror some of it, which could cost performance.

For WebGL, we could probably just fire the "webglcontextlost" and
"webglcontextrestored" events at page deserialization.
Losing all native state is an event planned by the specification (it
can happen when the screen resolution changes, power-related events,
and so on).
The web app is then responsible for restoring the state, typically
from from regular DOM resources (which can be serialized without
issue).


Regards,

Doug Turner

unread,
Sep 1, 2011, 12:59:07 AM9/1/11
to Mike Shaver, dev-pl...@lists.mozilla.org, Nicholas Nethercote

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

fwiw, we ran some simple perf tests using ashmem verses system malloc. We did not detect any significant performance penalties. There will be some overhead to "test" to ensure that the memory is still pinned, but that isn't going to be (or shouldn't be) in the hot path.

Doug


Robert O'Callahan

unread,
Sep 1, 2011, 1:25:10 AM9/1/11
to Cedric Vivier, Boris Zbarsky, dev-pl...@lists.mozilla.org, Mike Shaver
On Thu, Sep 1, 2011 at 4:55 PM, Cedric Vivier <cvi...@mozilla.com> wrote:

> On Thu, Sep 1, 2011 at 08:29, Robert O'Callahan <rob...@ocallahan.org>
> wrote:
> > WebGL would be interesting. If you can't get all the state out of the
> native
> > context, you'll need to mirror some of it, which could cost performance.
>
> For WebGL, we could probably just fire the "webglcontextlost" and
> "webglcontextrestored" events at page deserialization.
> Losing all native state is an event planned by the specification (it
> can happen when the screen resolution changes, power-related events,
> and so on).
>

Ugh, that's horrible :-). I bet most Web apps will completely fail to handle
that properly.

Jonas Sicking

unread,
Sep 1, 2011, 1:36:41 PM9/1/11
to rob...@ocallahan.org, Cedric Vivier, Boris Zbarsky, dev-pl...@lists.mozilla.org, Mike Shaver
On Wed, Aug 31, 2011 at 10:25 PM, Robert O'Callahan

<rob...@ocallahan.org> wrote:
> On Thu, Sep 1, 2011 at 4:55 PM, Cedric Vivier <cvi...@mozilla.com> wrote:
>
>> On Thu, Sep 1, 2011 at 08:29, Robert O'Callahan <rob...@ocallahan.org>
>> wrote:
>> > WebGL would be interesting. If you can't get all the state out of the
>> native
>> > context, you'll need to mirror some of it, which could cost performance.
>>
>> For WebGL, we could probably just fire the "webglcontextlost" and
>> "webglcontextrestored" events at page deserialization.
>> Losing all native state is an event planned by the specification (it
>> can happen when the screen resolution changes, power-related events,
>> and so on).
>>
>
> Ugh, that's horrible :-). I bet most Web apps will completely fail to handle
> that properly.

Agreed! I know I've never seen that handled in the WebGL examples I've
looked at, nor have I handled it in the WebGL code I've written. In
fact, this is the first I've heard of these events.

Is there no way we can remember the context in the browser whenever
it's set and restore it as needed?

/ Jonas

Cedric Vivier

unread,
Sep 1, 2011, 1:54:40 PM9/1/11
to Jonas Sicking, Boris Zbarsky, dev-pl...@lists.mozilla.org, rob...@ocallahan.org, Mike Shaver
On Fri, Sep 2, 2011 at 01:36, Jonas Sicking <jo...@sicking.cc> wrote:
> Agreed! I know I've never seen that handled in the WebGL examples I've
> looked at, nor have I handled it in the WebGL code I've written. In
> fact, this is the first I've heard of these events.

It's not surprising as no browser currently implement these afaik.
For now if you lose a context you lose it forever.
Fortunately it rarely if ever happens on desktop (where WebGL users
currently are).


> Is there no way we can remember the context in the browser whenever
> it's set and restore it as needed?

IIRC the WebGL working group consensus was that in any but trivial use
cases it would be very expensive to do so, if even possible to do
fully (eg. renderbuffers..).

It is not ideal, yes, otoh context restoration is something that is
not unknown to 3D developers (especially on mobile but also Direct3D
iirc). Fortunately, WebGL frameworks should be able to handle this
transparently for most developers (because they work on a higher
abstraction level, with concept of "scenes" and such).

A future revision of the WebGL spec might expose this mechanism much
better to developers as it would be the basis for asynchronous WebGL
context creation (which has other benefits as well).


Regards,

Jonas Sicking

unread,
Sep 1, 2011, 3:16:54 PM9/1/11
to Cedric Vivier, Boris Zbarsky, dev-pl...@lists.mozilla.org, rob...@ocallahan.org, Mike Shaver
On Thu, Sep 1, 2011 at 10:54 AM, Cedric Vivier <cvi...@mozilla.com> wrote:
>> Is there no way we can remember the context in the browser whenever
>> it's set and restore it as needed?
>
> IIRC the WebGL working group consensus was that in any but trivial use
> cases it would be very expensive to do so, if even possible to do
> fully (eg. renderbuffers..).

Just because the WebGL WG felt that way doesn't mean that we have to
accept that. One of mozillas responsibilities has always been to fight
for good standards that help users and developers.

> It is not ideal, yes, otoh context restoration is something that is
> not unknown to 3D developers (especially on mobile but also Direct3D
> iirc). Fortunately, WebGL frameworks should be able to handle this
> transparently for most developers (because they work on a higher
> abstraction level, with concept of "scenes" and such).
>
> A future revision of the WebGL spec might expose this mechanism much
> better to developers as it would be the basis for asynchronous WebGL
> context creation (which has other benefits as well).

I think the question we need to ask ourselves are "how bad would it be
if no-one listened to these events", because that's the most realistic
scenario. If not listening to the event means that the WebGL canvas
will go blank every 5 seconds on some systems, then I don't think that
the WebGL spec can be thought of to work for those systems.

/ Jonas

Cedric Vivier

unread,
Sep 1, 2011, 3:41:36 PM9/1/11
to Jonas Sicking, Vladimir Vukicevic, Mike Shaver, Benoit Jacob, Boris Zbarsky, dev-pl...@lists.mozilla.org, rob...@ocallahan.org
On Fri, Sep 2, 2011 at 03:16, Jonas Sicking <jo...@sicking.cc> wrote:
> On Thu, Sep 1, 2011 at 10:54 AM, Cedric Vivier <cvi...@mozilla.com> wrote:
>>> Is there no way we can remember the context in the browser whenever
>>> it's set and restore it as needed?
>>
>> IIRC the WebGL working group consensus was that in any but trivial use
>> cases it would be very expensive to do so, if even possible to do
>> fully (eg. renderbuffers..).
>
> Just because the WebGL WG felt that way doesn't mean that we have to
> accept that. One of mozillas responsibilities has always been to fight
> for good standards that help users and developers.


Mozilla is participating in the WebGL WG... Many pros and cons have
been debated about this.
CC'ing Benoit and Vlad (who was editor of the spec at the time this
issue has been discussed iirc)


>
>> It is not ideal, yes, otoh context restoration is something that is
>> not unknown to 3D developers (especially on mobile but also Direct3D
>> iirc). Fortunately, WebGL frameworks should be able to handle this
>> transparently for most developers (because they work on a higher
>> abstraction level, with concept of "scenes" and such).
>>
>> A future revision of the WebGL spec might expose this mechanism much
>> better to developers as it would be the basis for asynchronous WebGL
>> context creation (which has other benefits as well).

If an app is not listening to the event, the context is invalidated
once and for all. Not "every 5 seconds".

A hypothetical system losing a native 3D context so often is just not
gonna work at all, regardless of WebGL.


Regards,

Benoit Jacob

unread,
Sep 1, 2011, 4:03:07 PM9/1/11
to Cedric Vivier, Boris Zbarsky, Mike Shaver, dev-pl...@lists.mozilla.org, Jonas Sicking, rob...@ocallahan.org
2011/9/1 Cedric Vivier <cvi...@mozilla.com>:

> On Fri, Sep 2, 2011 at 01:36, Jonas Sicking <jo...@sicking.cc> wrote:
>> Agreed! I know I've never seen that handled in the WebGL examples I've
>> looked at, nor have I handled it in the WebGL code I've written. In
>> fact, this is the first I've heard of these events.
>
> It's not surprising as no browser currently implement these afaik.

Chrome does implement that.

But still, it's not surprising that most web pages don't care about
it. The only case where it becomes very important in practice, is on
cell phones where the contexts get lost everytime the phone goes to
sleep (I believe). Which is also why only OpenGL ES / EGL, not desktop
OpenGL, has provisions for "losing" the context.

Since most pages don't take care of that, and that's likely not
changing, we can't rely on that.

>> Is there no way we can remember the context in the browser whenever
>> it's set and restore it as needed?
>
> IIRC the WebGL working group consensus was that in any but trivial use
> cases it would be very expensive to do so, if even possible to do
> fully (eg. renderbuffers..).

We can't cache all WebGL state as soon as it's set for 2 reasons:
- some WebGL state such as the contents of textures and renderbuffers
can be set by shaders on the GPU, and we don't want to round trip to
the CPU and main memory everytime the GPU touches them, as that would
kill performance
- anyhow that would double WebGL memory usage which is the opposite
of a MemShrink goal

As for saving a copy of all WebGL state at the last minute, not only
it's very hard to do (without even a guarantee that it's possible) but
it would NOT be "discarding regenerable stuff", this would just be
moving it from one place (video or main memory allocated by the OpenGL
implementation) to another place (copies in main memory in our address
space). I don't see the benefit.

I agree that at this point the only approach to "discarding
regenerable stuff" is to rely on WebGL lost context events being
correctly handled by every web page but that doesn't seem realistic at
this point. Say that less that 1% of current WebGL pages handle these
events, and we'd need to be at, say, 99% for this solution to be
practical, I don't know how we go from 1% to 99%.

Benoit

Benoit Jacob

unread,
Sep 1, 2011, 4:15:09 PM9/1/11
to Cedric Vivier, Vladimir Vukicevic, Mike Shaver, rob...@ocallahan.org, Benoit Jacob, Boris Zbarsky, dev-pl...@lists.mozilla.org, Jonas Sicking
2011/9/1 Cedric Vivier <cvi...@mozilla.com>:

> On Fri, Sep 2, 2011 at 03:16, Jonas Sicking <jo...@sicking.cc> wrote:
>> On Thu, Sep 1, 2011 at 10:54 AM, Cedric Vivier <cvi...@mozilla.com> wrote:
>>>> Is there no way we can remember the context in the browser whenever
>>>> it's set and restore it as needed?
>>>
>>> IIRC the WebGL working group consensus was that in any but trivial use
>>> cases it would be very expensive to do so, if even possible to do
>>> fully (eg. renderbuffers..).
>>
>> Just because the WebGL WG felt that way doesn't mean that we have to
>> accept that. One of mozillas responsibilities has always been to fight
>> for good standards that help users and developers.

I must have missed something. All I see above is "Is there no way we
can remember the context in the browser wheneve it's set and restore
it as needed?" which just doesn't work for reasons explained in my
previous email; that's not WebGL specific, not even OpenGL
specific,it's just that "state" includes things like texture and
renderbuffer contents which you really, really don't want to cache
everytime they're modified, be it by a WebGL command running on the
CPU or by a shader running on the GPU.

Benoit

>
>
> Mozilla is participating in the WebGL WG... Many pros and cons have
> been debated about this.
> CC'ing Benoit and Vlad (who was editor of the spec at the time this
> issue has been discussed iirc)
>
>
>>
>>> It is not ideal, yes, otoh context restoration is something that is
>>> not unknown to 3D developers (especially on mobile but also Direct3D
>>> iirc). Fortunately, WebGL frameworks should be able to handle this
>>> transparently for most developers (because they work on a higher
>>> abstraction level, with concept of "scenes" and such).
>>>
>>> A future revision of the WebGL spec might expose this mechanism much
>>> better to developers as it would be the basis for asynchronous WebGL
>>> context creation (which has other benefits as well).
>
> If an app is not listening to the event, the context is invalidated
> once and for all. Not "every 5 seconds".
>
> A hypothetical system losing a native 3D context so often is just not
> gonna work at all, regardless of WebGL.
>
>
> Regards,

Cedric Vivier

unread,
Sep 1, 2011, 4:14:46 PM9/1/11
to Benoit Jacob, Boris Zbarsky, Mike Shaver, dev-pl...@lists.mozilla.org, Jonas Sicking, rob...@ocallahan.org
On Fri, Sep 2, 2011 at 04:03, Benoit Jacob <jacob.b...@gmail.com> wrote:
> I agree that at this point the only approach to "discarding
> regenerable stuff" is to rely on WebGL lost context events being
> correctly handled by every web page but that doesn't seem realistic at
> this point. Say that less that 1% of current WebGL pages handle these
> events, and we'd need to be at, say, 99% for this solution to be
> practical, I don't know how we go from 1% to 99%.

I'm more or less optimistic about this in the long-term (by the time
we have popular non-demo WebGL apps)
That will be tough but not impossible, I can see the situation
improving much when :

1) context restoration events are implemented by all browsers
2) they are easy(er) to test on desktop (having a "tab snapshot"
feature as discussed in this thread would be an example)
3) the eventsare much more visible to developers as the normal "entry
point" to start a WebGL context (asynchronous context creation in a
future WebGL revision)
4) have WebGL finally available on many mobile devices (and not only
an insignificant tiny fraction of them as it is now). People will
steer away from WebGL apps that they cannot use when they come back
from standby on their phone/tablet. Making supporting them a
requirement for any serious WebGL developer.


Regards,

Robert O'Callahan

unread,
Sep 1, 2011, 6:07:18 PM9/1/11
to Cedric Vivier, Benoit Jacob, Boris Zbarsky, dev-pl...@lists.mozilla.org, Jonas Sicking, Mike Shaver
On Fri, Sep 2, 2011 at 8:14 AM, Cedric Vivier <cvi...@mozilla.com> wrote:

> 1) context restoration events are implemented by all browsers
> 2) they are easy(er) to test on desktop (having a "tab snapshot"
> feature as discussed in this thread would be an example)
> 3) the eventsare much more visible to developers as the normal "entry
> point" to start a WebGL context (asynchronous context creation in a
> future WebGL revision)
> 4) have WebGL finally available on many mobile devices (and not only
> an insignificant tiny fraction of them as it is now). People will
> steer away from WebGL apps that they cannot use when they come back
> from standby on their phone/tablet. Making supporting them a
> requirement for any serious WebGL developer.
>

Sounds reasonable. I understand that making context restoration work
automatically could be infeasible.

0 new messages