Event Pages

[nsm.n...@gmail.com]

Hi everyone,

(Background: [1], [2], [3])
(Tracking bug: https://bugzil.la/903441)

Here are my thoughts after experimenting with SharedWorker code and writing the first bits of the event page code. Feedback is appreciated before I proceed with more work. What I've got now is on the bug. Just a note, SharedWorker hasn't landed yet.

The core idea is to have a worker like thing, that can run a script in a separate thread, but unlike SharedWorkers, the lifetime of the script is not bound to the lifetime of windows that hold a reference to it.

In addition, it should be possible to start this script on its own, from chrome code, on the basis of certain information supplied by another WebAPI, or from the manifest in case of apps.

Event Pages or Background Service?
----------------------------------
The reason I ask is Event pages are something that are short lived. But the moment such a page is allowed to have MessagePorts to user visible web pages, or can be created by content code, they'll be long running. Should this be allowed or should we enforce a distinction and require that developers use SharedWorkers for the later case?

In addition, if NavigationController intends to use these, they may or may not be long running depending on what is requested for download (a video file for example).

I see no technical reason not to allow long running pages, I just want to clear the nomenclature.

A copy of my local notes:
-------------------------

A Event Page:
- Can run with no user visible normal HTML pages open.
- Can run with zero MessagePorts to the above pages.
- Runs in the same process as the application (on B2G)
- Separate thread. It *is* a Worker as far as implementation goes, so it inherits all those properties.
- Should be able to query user agent for currently user-visible same-origin pages that are open.
- Should receive events when user-visible same-origin pages open or close, with MessagePorts to them (SharedWorker onconnect, perhaps supplemented with ondisconnect?)
- Should be able to *launch* a new tab/app (This will need to be done right, and require UX support, and possibly another permission)
- Should be able to be started by the platform
- Should be able to receive system messages

Event Pages are killed/unloaded when they've been idle (no incoming events) for a small interval (of course, there should be no windows holding references to it.) So XHR and timers will hold pages alive, as will other async APIs that get ported to workers. An event page may also call close() on itself (from the Worker spec).

In addition, an event page may be immediately unloaded/suspended if:
* Low memory and it is not part of the foreground app.
* Process priority is lowered by scheduler
These two are likely to apply to B2G only, but will require thought about how the process priority is affected (Prior art [4]).

Based on the current SharedWorker spec and patch [5], much of the common ground is implemented, the difference is:
* Event Page lifecycle is not bound strongly to windows, while SharedWorker's is.
* An Event Page, unlike workers, is *not suspended/resumed* based on all refering windows being suspended/resumed. That is, event page timers will not stop even if all DOM windows that hold a reference to it are suspended, while SharedWorker's will be.

APIs that require event pages or find them extremely useful:
SimplePush, Alarms, maybe various device information receivers (battery, bluetooth)
This kind of 'background' capability has been requested by developers within and outside Mozilla.

APIs that are needed by Event Pages (that is, they need to be ported to workers): system messages, desktop notifications (So pages/apps can show notifications without opening a tab/window).

Cheers,
Nikhil

[1]: https://groups.google.com/forum/#!topic/mozilla.dev.platform/un6c6sN6KwA
[2]: https://groups.google.com/d/msg/mozilla.dev.webapi/diddFoHYLVs/4zOOC1MoJrMJ
[3]: http://developer.chrome.com/extensions/event_pages.html
[4]: https://developer.android.com/reference/android/app/Service.html#ProcessLifecycle
[5]: https://bugzil.la/643325

[nsm.n...@gmail.com]

On Monday, August 12, 2013 6:25:41 PM UTC-7, nsm.n...@gmail.com wrote:
>
> - Should be able to *launch* a new tab/app (This will need to be done right, and require UX support, and possibly another permission)

To add to this, it may be better to add a thin wrapper around desktop notifications, which accepts notification parameters and a URL, and only loads the URL when the user clicks/taps the notification. This prevents the page from opening tabs without user approval.

Nikhil

[nsm.n...@gmail.com]

On Monday, August 12, 2013 6:25:41 PM UTC-7, nsm.n...@gmail.com wrote:
>

> Event Pages are killed/unloaded when they've been idle (no incoming events) for a small interval (of course, there should be no windows holding references to it.) So XHR and timers will hold pages alive, as will other async APIs that get ported to workers. An event page may also call close() on itself (from the Worker spec).
>
>

Another issue is if event pages should be allowed to use setInterval() or long setTimeout() to keep themselves alive.

Nikhil

[Ehsan Akhgari]

Thanks a lot for starting this conversation!

On 2013-08-12 9:25 PM, nsm.n...@gmail.com wrote:
> Hi everyone,
>
> (Background: [1], [2], [3])
> (Tracking bug: https://bugzil.la/903441)
>
> Here are my thoughts after experimenting with SharedWorker code and writing the first bits of the event page code. Feedback is appreciated before I proceed with more work. What I've got now is on the bug. Just a note, SharedWorker hasn't landed yet.
>
> The core idea is to have a worker like thing, that can run a script in a separate thread, but unlike SharedWorkers, the lifetime of the script is not bound to the lifetime of windows that hold a reference to it.
>
> In addition, it should be possible to start this script on its own, from chrome code, on the basis of certain information supplied by another WebAPI, or from the manifest in case of apps.
>
> Event Pages or Background Service?
> ----------------------------------
> The reason I ask is Event pages are something that are short lived. But the moment such a page is allowed to have MessagePorts to user visible web pages, or can be created by content code, they'll be long running. Should this be allowed or should we enforce a distinction and require that developers use SharedWorkers for the later case?

I think we should try to make the lifetime of the page controllable by
the UA if possible. In other words, the event page should expect to get
killed by the UA at any time. We can do that by breaking the
MessagePort references to the event page held by normal content pages
when the event page goes away. I think it would be a very bad idea to
make those references strong, since that would mean that if the author
forgets to release those references when they no longer need the event
page, we may render ourselves unable to kill the event page.

> In addition, if NavigationController intends to use these, they may or may not be long running depending on what is requested for download (a video file for example).

For NavigationController, the controller should assume that it will get
killed after handling every single request. In fact I'm planning to add
that as a debugging option to catch content which has assumptions about
its lifetime.

> I see no technical reason not to allow long running pages, I just want to clear the nomenclature.

If we leave the lifetime management of the event pages to the UA, we can
make intelligent choices on when to kill the event page based on things
like whether there are MessagePorts accessing it, whether it belongs to
a foreground/background app, whether we're low on system resources, etc.
Therefore, event pages may or may not be long running, I don't see why
we should clarify it further.

> A copy of my local notes:
> -------------------------
>
> A Event Page:
> - Can run with no user visible normal HTML pages open.
> - Can run with zero MessagePorts to the above pages.
> - Runs in the same process as the application (on B2G)

One thing that is not clear to me is the origin/process mapping. How do
you plan to handle the case where two processes render same origin
content, and they want to access a shared event page? Note that this
could happen both in Firefox OS and on desktop Firefox with e10s.

> - Separate thread. It *is* a Worker as far as implementation goes, so it inherits all those properties.
> - Should be able to query user agent for currently user-visible same-origin pages that are open.
> - Should receive events when user-visible same-origin pages open or close, with MessagePorts to them (SharedWorker onconnect, perhaps supplemented with ondisconnect?)
> - Should be able to *launch* a new tab/app (This will need to be done right, and require UX support, and possibly another permission)
> - Should be able to be started by the platform
> - Should be able to receive system messages

Another issue, did we decide that we want event pages to run in a worker
context, as opposed to a regular `window' context?

> Event Pages are killed/unloaded when they've been idle (no incoming events) for a small interval (of course, there should be no windows holding references to it.) So XHR and timers will hold pages alive, as will other async APIs that get ported to workers. An event page may also call close() on itself (from the Worker spec).

This doesn't seem like what we want. What if the event page creates a
timer but never destroys it? The way this paragraph is phrased it would
seem like we could never kill such a page.

> In addition, an event page may be immediately unloaded/suspended if:
> * Low memory and it is not part of the foreground app.
> * Process priority is lowered by scheduler
> These two are likely to apply to B2G only, but will require thought about how the process priority is affected (Prior art [4]).

Agreed, however I think we should hide these heuristics under the "UA
gets the say on the lifetime" rule.

> Based on the current SharedWorker spec and patch [5], much of the common ground is implemented, the difference is:
> * Event Page lifecycle is not bound strongly to windows, while SharedWorker's is.
> * An Event Page, unlike workers, is *not suspended/resumed* based on all refering windows being suspended/resumed. That is, event page timers will not stop even if all DOM windows that hold a reference to it are suspended, while SharedWorker's will be.

I wanted to ask clarification on this second point, but that sort of
depends on your answer to the timer issue I raised above. :-)

Cheers,
Ehsan

> APIs that require event pages or find them extremely useful:
> SimplePush, Alarms, maybe various device information receivers (battery, bluetooth)
> This kind of 'background' capability has been requested by developers within and outside Mozilla.
>
> APIs that are needed by Event Pages (that is, they need to be ported to workers): system messages, desktop notifications (So pages/apps can show notifications without opening a tab/window).
>
> Cheers,
> Nikhil
>
> [1]: https://groups.google.com/forum/#!topic/mozilla.dev.platform/un6c6sN6KwA
> [2]: https://groups.google.com/d/msg/mozilla.dev.webapi/diddFoHYLVs/4zOOC1MoJrMJ
> [3]: http://developer.chrome.com/extensions/event_pages.html
> [4]: https://developer.android.com/reference/android/app/Service.html#ProcessLifecycle
> [5]: https://bugzil.la/643325

> _______________________________________________
> dev-webapi mailing list
> dev-w...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-webapi
>

[nsm.n...@gmail.com]

On Wednesday, August 14, 2013 8:27:48 AM UTC-7, Ehsan Akhgari wrote:
>
> > Event Pages or Background Service?
>
> > ----------------------------------
>
> > The reason I ask is Event pages are something that are short lived. But the moment such a page is allowed to have MessagePorts to user visible web pages, or can be created by content code, they'll be long running. Should this be allowed or should we enforce a distinction and require that developers use SharedWorkers for the later case?
>
>
>
> I think we should try to make the lifetime of the page controllable by
>
> the UA if possible. In other words, the event page should expect to get
>
> killed by the UA at any time. We can do that by breaking the
>
> MessagePort references to the event page held by normal content pages
>
> when the event page goes away. I think it would be a very bad idea to
>
> make those references strong, since that would mean that if the author
>
> forgets to release those references when they no longer need the event
>
> page, we may render ourselves unable to kill the event page.

I agree that the lifetime should be UA controlled. But how do we do it without exposing the concept of weak references to content code? That is I'm not aware of prior art on this and how good an idea it is that a port suddenly refers to nothing. One suggestion is to add an 'close' event to MessagePort that is fired before the port is closed because the event page is killed.

>

>
> If we leave the lifetime management of the event pages to the UA, we can
>
> make intelligent choices on when to kill the event page based on things
>
> like whether there are MessagePorts accessing it, whether it belongs to
>
> a foreground/background app, whether we're low on system resources, etc.
>
> Therefore, event pages may or may not be long running, I don't see why
>
> we should clarify it further.

I just think that we should call them something other than Event Page if they are going to be long running, like Service ;)

>
>
>
> > A copy of my local notes:
>
> > -------------------------
>
> >
>
> > A Event Page:
>
> > - Can run with no user visible normal HTML pages open.
>
> > - Can run with zero MessagePorts to the above pages.
>
> > - Runs in the same process as the application (on B2G)
>
>
>
> One thing that is not clear to me is the origin/process mapping. How do
>
> you plan to handle the case where two processes render same origin
>
> content, and they want to access a shared event page? Note that this
>
> could happen both in Firefox OS and on desktop Firefox with e10s.

Considering that MessagePorts abstract away the communication barrier and make it async, the event page could easily work cross process. In fact, if we want to allow some sort of system where b2g apps can talk to services from other apps, it would definitely need IPC support, but this would introduce significant complexity in keeping things alive, and I haven't really thought through something like this yet, which is why I'm sticking to same process event pages. From what I read of the current SharedWorker code, it does not attempt to deal with this either. I'm all for any insight anyone has to offer about dealing with this, since I'm not as experienced with Gecko internals, nor with the IPC model.

>
> Another issue, did we decide that we want event pages to run in a worker
>
> context, as opposed to a regular `window' context?

It was the decision in a face to face me and Jonas had, and it does allow a lot of code re-use. With work to get more DOM APIs onto Workers, I think its a fair decision.

>
>
>
> > Event Pages are killed/unloaded when they've been idle (no incoming events) for a small interval (of course, there should be no windows holding references to it.) So XHR and timers will hold pages alive, as will other async APIs that get ported to workers. An event page may also call close() on itself (from the Worker spec).
>
>
>
> This doesn't seem like what we want. What if the event page creates a
>
> timer but never destroys it? The way this paragraph is phrased it would
>
> seem like we could never kill such a page.

Jonas and me haven't thought of a good solution to this yet. With sufficient documentation stating NOT to do this and the use of the Alarm API, it should not be a problem for well meaning applications. So I guess we just ignore pending timers when UA decides to kill an event page. But certain things like pending IDB events we would have to wait for, so as not to end up in an inconsistent DB state or something. I guess this is hidden under the "UA in control" criterion.

>
>
>
> > In addition, an event page may be immediately unloaded/suspended if:
>
> > * Low memory and it is not part of the foreground app.
>
> > * Process priority is lowered by scheduler
>
> > These two are likely to apply to B2G only, but will require thought about how the process priority is affected (Prior art [4]).
>
>
>
> Agreed, however I think we should hide these heuristics under the "UA
>
> gets the say on the lifetime" rule.

Of course, just noted it here so that someone could catch me if this wasn't implemented.

Nikhil

[Ehsan Akhgari]

On 2013-08-15 5:14 PM, nsm.n...@gmail.com wrote:
> On Wednesday, August 14, 2013 8:27:48 AM UTC-7, Ehsan Akhgari wrote:
>>
>>> Event Pages or Background Service?
>>
>>> ----------------------------------
>>
>>> The reason I ask is Event pages are something that are short lived. But the moment such a page is allowed to have MessagePorts to user visible web pages, or can be created by content code, they'll be long running. Should this be allowed or should we enforce a distinction and require that developers use SharedWorkers for the later case?
>>
>>
>>
>> I think we should try to make the lifetime of the page controllable by
>>
>> the UA if possible. In other words, the event page should expect to get
>>
>> killed by the UA at any time. We can do that by breaking the
>>
>> MessagePort references to the event page held by normal content pages
>>
>> when the event page goes away. I think it would be a very bad idea to
>>
>> make those references strong, since that would mean that if the author
>>
>> forgets to release those references when they no longer need the event
>>
>> page, we may render ourselves unable to kill the event page.
>
> I agree that the lifetime should be UA controlled. But how do we do it without exposing the concept of weak references to content code? That is I'm not aware of prior art on this and how good an idea it is that a port suddenly refers to nothing. One suggestion is to add an 'close' event to MessagePort that is fired before the port is closed because the event page is killed.

That is a great question. I wonder if we can use the same kind of trick
that we use in order to break references to objects from other documents
when we kill those documents? Hopefully Kyle knows.

>> If we leave the lifetime management of the event pages to the UA, we can
>>
>> make intelligent choices on when to kill the event page based on things
>>
>> like whether there are MessagePorts accessing it, whether it belongs to
>>
>> a foreground/background app, whether we're low on system resources, etc.
>>
>> Therefore, event pages may or may not be long running, I don't see why
>>
>> we should clarify it further.
>
> I just think that we should call them something other than Event Page if they are going to be long running, like Service ;)

Yes, event pages is a terrible name IMO. :-) BackgroundService is
better if you ask me.

>>> A copy of my local notes:
>>
>>> -------------------------
>>
>>>
>>
>>> A Event Page:
>>
>>> - Can run with no user visible normal HTML pages open.
>>
>>> - Can run with zero MessagePorts to the above pages.
>>
>>> - Runs in the same process as the application (on B2G)
>>
>>
>>
>> One thing that is not clear to me is the origin/process mapping. How do
>>
>> you plan to handle the case where two processes render same origin
>>
>> content, and they want to access a shared event page? Note that this
>>
>> could happen both in Firefox OS and on desktop Firefox with e10s.
>
> Considering that MessagePorts abstract away the communication barrier and make it async, the event page could easily work cross process. In fact, if we want to allow some sort of system where b2g apps can talk to services from other apps, it would definitely need IPC support, but this would introduce significant complexity in keeping things alive, and I haven't really thought through something like this yet, which is why I'm sticking to same process event pages. From what I read of the current SharedWorker code, it does not attempt to deal with this either. I'm all for any insight anyone has to offer about dealing with this, since I'm not as experienced with Gecko internals, nor with the IPC model.

I think we should be able to use proxy-like objects that do the right
thing when their weak reference to the other side is dead. I'm not sure
about all of the IPC semantics that we'll need to implement this, but it
doesn't seem too hard. As long as we make sure we're not digging
ourselves in a hole here we can worry about the implementation of this
part later.

>> Another issue, did we decide that we want event pages to run in a worker
>>
>> context, as opposed to a regular `window' context?
>
> It was the decision in a face to face me and Jonas had, and it does allow a lot of code re-use. With work to get more DOM APIs onto Workers, I think its a fair decision.

OK.

>>> Event Pages are killed/unloaded when they've been idle (no incoming events) for a small interval (of course, there should be no windows holding references to it.) So XHR and timers will hold pages alive, as will other async APIs that get ported to workers. An event page may also call close() on itself (from the Worker spec).
>>
>>
>>
>> This doesn't seem like what we want. What if the event page creates a
>>
>> timer but never destroys it? The way this paragraph is phrased it would
>>
>> seem like we could never kill such a page.
>
> Jonas and me haven't thought of a good solution to this yet. With sufficient documentation stating NOT to do this and the use of the Alarm API, it should not be a problem for well meaning applications. So I guess we just ignore pending timers when UA decides to kill an event page. But certain things like pending IDB events we would have to wait for, so as not to end up in an inconsistent DB state or something. I guess this is hidden under the "UA in control" criterion.

This may be difficult to scale in the future as we port more APIs to
workers. But yes, as long as the UA controls the lifetime, we can
decide to do anything we need to here.


Cheers,
Ehsan

[Kyle Huey]

On Fri, Aug 16, 2013 at 8:08 AM, Ehsan Akhgari <ehsan....@gmail.com>wrote:

> On 2013-08-15 5:14 PM, nsm.n...@gmail.com wrote:
>

>> On Wednesday, August 14, 2013 8:27:48 AM UTC-7, Ehsan Akhgari wrote:
>>
>>>
>>> Event Pages or Background Service?
>>>>
>>>

>>> ------------------------------**----

>>>>
>>>
>>> The reason I ask is Event pages are something that are short lived. But
>>>> the moment such a page is allowed to have MessagePorts to user visible web
>>>> pages, or can be created by content code, they'll be long running. Should
>>>> this be allowed or should we enforce a distinction and require that
>>>> developers use SharedWorkers for the later case?
>>>>
>>>
>>>
>>>
>>> I think we should try to make the lifetime of the page controllable by
>>>
>>> the UA if possible. In other words, the event page should expect to get
>>>
>>> killed by the UA at any time. We can do that by breaking the
>>>
>>> MessagePort references to the event page held by normal content pages
>>>
>>> when the event page goes away. I think it would be a very bad idea to
>>>
>>> make those references strong, since that would mean that if the author
>>>
>>> forgets to release those references when they no longer need the event
>>>
>>> page, we may render ourselves unable to kill the event page.
>>>
>>
>> I agree that the lifetime should be UA controlled. But how do we do it
>> without exposing the concept of weak references to content code? That is
>> I'm not aware of prior art on this and how good an idea it is that a port
>> suddenly refers to nothing. One suggestion is to add an 'close' event to
>> MessagePort that is fired before the port is closed because the event page
>> is killed.
>>
>

> That is a great question. I wonder if we can use the same kind of trick
> that we use in order to break references to objects from other documents
> when we kill those documents? Hopefully Kyle knows.
>

Perhaps I'm missing something, but I don't see why you need anything that
strong. A 'close' event on MessagePort that can be fired by the UA at any
time seems like the way to go. Our cross-compartment wrapper cutting stuff
is designed to deal with arbitrary JS references from one compartment to
another. In fact, it won't do anything at all here if I understand things
correctly, because there are no cross-compartment JS references (the C++
MessagePort mediates both ends).

(Also implementing something like that would be quite difficult in WebKit
since they don't have wrappers between same-origin globals, though I think
they'll break down at some point and implement them for the security
benefits.)

- Kyle

[Ehsan Akhgari]

On 2013-08-19 10:56 AM, Kyle Huey wrote:
> On Fri, Aug 16, 2013 at 8:08 AM, Ehsan Akhgari <ehsan....@gmail.com
> <mailto:ehsan....@gmail.com>> wrote:
>
> On 2013-08-15 5:14 PM, nsm.n...@gmail.com

> <mailto:nsm.n...@gmail.com> wrote:
>
> On Wednesday, August 14, 2013 8:27:48 AM UTC-7, Ehsan Akhgari wrote:
>
>
> Event Pages or Background Service?
>
>

> ------------------------------__----

> That is a great question. I wonder if we can use the same kind of
> trick that we use in order to break references to objects from other
> documents when we kill those documents? Hopefully Kyle knows.
>
>
> Perhaps I'm missing something, but I don't see why you need anything
> that strong. A 'close' event on MessagePort that can be fired by the UA
> at any time seems like the way to go. Our cross-compartment wrapper
> cutting stuff is designed to deal with arbitrary JS references from one
> compartment to another. In fact, it won't do anything at all here if I
> understand things correctly, because there are no cross-compartment JS
> references (the C++ MessagePort mediates both ends).

Alright, sounds good!

> (Also implementing something like that would be quite difficult in
> WebKit since they don't have wrappers between same-origin globals,
> though I think they'll break down at some point and implement them for
> the security benefits.)

Hmm, that could be a problem...

Ehsan

[nsm.n...@gmail.com]

Apologies for forgetting to update this public thread a couple of weeks ago.

At that time, some Mozilla folks and some Google folks had a video conference about this. In that a couple of things were discussed:

* Navigation Controller and Service Workers [1] are effectively the same Worker like thing, with slightly different APIs available. A draft of what ServiceWorker registration may look like is [2]. In fact, it is possible for Navigation Controller to be a subset of Service Worker.
* Apps will need a manifest entry specifying this.
* The upgrade path and version and so on can be borrowed from Navigation Controller. I have no immediate plans to deal with this.
* NC has a wait() sort of method. The plan is to borrow this to Service Workers, as a form of acknowledgement that an event meant for the worker has been handled as intended by the worker. This is required because a Service Worker may be terminated at the UA's discretion and may have unhandled events.
* The worker has an ability to send messages to its same origin windows [2]
* As of now there is no way to 'create a Service Worker' from the DOM. I can think of a couple of use cases, but again I'm waiting a while.
* As a proof of concept, Mozilla is going to get mozAlarms working with ServiceWorker.

In light of this, updating bug 903441.

Nikhil

[1]: formerly Event Pages, Alex Russell's gist calls them ServiceWorker, which seems quite appropriate, and from now on, that is what I call them in code I write (see bug 903441), but I'll make the change 'global' once I'm convinced its the right term.
[2]: https://gist.github.com/slightlyoff/7fae65a908ac318f69a3

[nsm.n...@gmail.com]

Spec is now tracked at https://github.com/slightlyoff/ServiceWorker which supersedes NavigationController.