Improving system messages to support webpages and more

[Jonas Sicking]

Hi All,

We're currently struggling a bit with a couple of things for system
messages. For background on system messages, see the thread here

https://groups.google.com/forum/#!topic/mozilla.dev.webapi/o8bkwx0EtmM/discussion

However the current implementation of system messages only works well
for apps where either
* The app consists of only a single webpage.
* The app wants to always navigate the user to a specific page
whenever there's an incoming message of the given type.

In other cases the app risks navigating the user as soon as there's an
incoming message, whether handling that message needs UI or not.

Also, any time there's an incoming message, even if the app starts in
the background, the app shows up in the application switcher. This can
be confusing to users since they never started the app.

We also have the problem that since system messages rely on manifests,
they only work for apps currently, not for normal web pages. While
we're planning on making it possible for websites to create a manifest
for their web pages, forcing people to create a manifest just because
they want to handle a system message.

The two APIs where we currently could use system messages for web pages are:

* Push notifications. An incoming message from a server needs to be
handled by a webpage. In some cases by showing UI, in other cases in
the background.
* Web Notifications. This spec is what allows a webpage to put a
message in the "message center". This works fine as long as the page
is being displayed as long as the message sits around in the message
center. However the spec doesn't work well if the user leaves the page
before seeing the message in the message center. If the user clicks
the message at that point we currently can't notify the page.


To solve these problems I propose we start by changing the syntax in
the manifest to something like:

"messages": [
{ "alarm": { page: "/index.html" } }
]

We can then add support for

"messages": [
{ "alarm": { eventPage: "/event-page.html" } }
]

This borrows an idea from how Chrome OS is handling similar issues. In
Chrome OS all incoming messages are sent to a special "events page".
This is a normal DOM window, but one that isn't ever rendered. The
window has access to all the normal APIs, including window.open for
opening new, visible, windows. It would also need access to new API
for enumerating all currently opened windows.

When a system message is sent to the event page for an application we
first start the event page if it's not already running. The page can
then process the incoming message and decide to either display no UI,
navigate an already opened window, or open a new window. All depending
on application specific logic using information in the incoming
message and from elsewhere.

We could also eventually add support for something like:

"messages": [
{ "alarm": { worker: "/messageHandler.js" } }
]

Which would work essentially the same way except that instead of using
a window to handle the message, it'd use a worker. This mainly has the
advantage that certain APIs are only available to workers since they
rely on synchronous IO. This is likely not something that we're going
to implement very soon since we need additional plumbing support in
workers in order to be able to do this. Also, since we have fewer APIs
supported in workers it's not as attractive for developers yet.


In order to support system messages for web pages we can then borrow
from this syntax. So for example registering for a push notification
from a web page could look like:

navigator.push.register({ page: "/foo.html"});

or

navigator.push.register({ eventPage: "/foo.html"});

I.e. the syntax is the same as what's put as the value part in the
manifest. The first syntax would mean that when there's an incoming
event, we'd open a new tab with "/foo.html" and render to the user.
We'd then send that page a system message using the same syntax as we
currently use for system messages.

The second syntax would mean that a background page would be opened
and a system message would be sent to that page. If the page decides
to, it could then use window.open in order to open a new tab which is
displayed to the user.

Thoughts?

/ Jonas

[Justin Lebar]

While we're modifying system messages, another issue we need to
consider is how to deal with apps which get killed while handling a
system message.

In general, an app can be killed at any point in time. This is
particularly true for apps which handle system messages, since they
tend to run in the background, and so can be OOMed by either the main
process or the foreground app.

The only way I'm aware of to handle this properly is to make system
messages idempotent. That is, we must change the semantics of system
messages such that if an app receives a system message twice, that has
the same effect as receiving it once.

We probably also need to add "commit" semantics to system messages: An
app would "commit" a system message after it's completely handled it,
and the main process could then stop re-delivering this system
message.

Note that we cannot guarantee that after an app calls commit(), it
will never have that same system message delivered again, because the
app might get killed while running commit(). In general, we suffer
from the Byzantine Generals problem here, so from the app's
perspective, commit() is necessary but not sufficient to stop
receiving a given message.

-Justin

> _______________________________________________
> dev-webapi mailing list
> dev-w...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-webapi

[Jonas Sicking]

On Mon, May 6, 2013 at 12:57 PM, Justin Lebar <justin...@gmail.com> wrote:
> While we're modifying system messages, another issue we need to
> consider is how to deal with apps which get killed while handling a
> system message.
>
> In general, an app can be killed at any point in time. This is
> particularly true for apps which handle system messages, since they
> tend to run in the background, and so can be OOMed by either the main
> process or the foreground app.
>
> The only way I'm aware of to handle this properly is to make system
> messages idempotent. That is, we must change the semantics of system
> messages such that if an app receives a system message twice, that has
> the same effect as receiving it once.
>
> We probably also need to add "commit" semantics to system messages: An
> app would "commit" a system message after it's completely handled it,
> and the main process could then stop re-delivering this system
> message.
>
> Note that we cannot guarantee that after an app calls commit(), it
> will never have that same system message delivered again, because the
> app might get killed while running commit(). In general, we suffer
> from the Byzantine Generals problem here, so from the app's
> perspective, commit() is necessary but not sufficient to stop
> receiving a given message.

Yes. All good points.

I agree that we need to add a commit() function. A few related questions:

* Should we up an apps OOM priority until it has called commit()? At
least above other background apps.
* Do we need to worry about apps abusing this to prevent themselves
from getting OOM-killed?
* How do we prevent CPU-intensive loops where an app is repeatedly
killed and then restarted. Some sort of backoff seems needed.

/ Jonas

[Justin Lebar]

> I agree that we need to add a commit() function. A few related questions:

Just to be clear, commit() alone does not solve the question of
idempotence; I don't want us to gloss over that. We have to modify
all of the messages (or at least their semantics) so they're safe to
re-deliver.

> * Should we up an apps OOM priority until it has called commit()? At
> least above other background apps.

Right now, we give apps higher OOM priority when running their system
message handlers. An app can then request higher OOM priority after
that with a wake lock. I think this is probably fine for now.

> * Do we need to worry about apps abusing this to prevent themselves
> from getting OOM-killed?

An app can just play a silent audio file and automatically get a
pretty high OOM priority.

> * How do we prevent CPU-intensive loops where an app is repeatedly
> killed and then restarted. Some sort of backoff seems needed.

Sure.

[Jonas Sicking]

On Mon, May 6, 2013 at 6:23 PM, Justin Lebar <justin...@gmail.com> wrote:
>> I agree that we need to add a commit() function. A few related questions:
>
> Just to be clear, commit() alone does not solve the question of
> idempotence; I don't want us to gloss over that. We have to modify
> all of the messages (or at least their semantics) so they're safe to
> re-deliver.

Definitely.

>> * Should we up an apps OOM priority until it has called commit()? At
>> least above other background apps.
>
> Right now, we give apps higher OOM priority when running their system
> message handlers. An app can then request higher OOM priority after
> that with a wake lock. I think this is probably fine for now.
>
>> * Do we need to worry about apps abusing this to prevent themselves
>> from getting OOM-killed?
>
> An app can just play a silent audio file and automatically get a
> pretty high OOM priority.

Good point, that sounds indeed fine for now.

/ Jonas

[Gene Lian]

Please see my in-line comments. Looping the b2g mailing list since some of the discussions might not be highly related to the point of view of Web APIs.

----- Original Message -----
> From: "Justin Lebar" <justin...@gmail.com>
> To: "Jonas Sicking" <jo...@sicking.cc>
> Cc: "dev-webapi" <dev-w...@lists.mozilla.org>, "William Chen" <wc...@mozilla.com>, "Anne van Kesteren"
> <avanke...@mozilla.com>, "Doug Turner" <do...@mozilla.com>, "Mounir Lamouri" <mou...@mozilla.com>, "EDUARDO
> FULLEA CARRERA" <e...@tid.es>, dev-...@lists.mozilla.org
> Sent: Tuesday, May 7, 2013 3:57:48 AM
> Subject: Re: Improving system messages to support webpages and more
>
> We probably also need to add "commit" semantics to system messages:
> An
> app would "commit" a system message after it's completely handled it,
> and the main process could then stop re-delivering this system
> message.

In our current design, we do support an acknowledgement mechanism like the "commit()" you're talking about here. That is, the content process will internally send an acknowledgement back to the main process to clean up the pending message whenever the content process successfully receives a system message (please search "SystemMessageManager:Message:Return:OK" in SystemMessageManager.js). Let me try to elaborate more regarding our system message behaviours as below:

When the main process attempts to fire system message, it will do the following 3 things:

1. Deliver the system message to the app (if it's alive).
2. Keep a copy of the system message in the pending queue (no matter the app is alive or not).
3. Try to launch the app (no matter it's alive or not).

So we can discuss the following two scenarios before delivering the system message:

A. The app (content process) is alive:

The main process knows the app is running (at least at that moment) so it fires system message to the target content process (step #1). When the content process receives the system message, it will immediately fire an acknowledgement back to the main process. The main process will then clean up the pending message (queued by step #2) because it has been successfully delivered for sure. Note that the attempt of opening the app (step #3) doesn't really help because the app has already been running. However, we could still have an edge case that the app can probably be killed (due to OOM'ed) before receiving the system message or sending the acknowledgement back. Under this circumstance, the step #3 will eventually take effect and launch the app, then it will go to the following case B.

B. The app (content process) is NOT alive:

Obviously, the main process won't fire the system message to the content process because the main process understands the target isn't alive. Anyway, the main process will keep the system message in the pending queue (step #2). Later, the step #3 will then launch the app in the long run. When the launching app calls .mozSetMessageHandler(...), it will try to retrieve all the pending system messages from the central and clean them up at the same time [1]. If the app is launched manually by users instead of being launched by step #3, everything still goes the same except that the step #3 doesn't take effect.

[1] A potential bug comes into my mind that we should also fire an acknowledgement back to clean up the system message, instead of cleaning it up during the retrieving process, because the retrieving process is an async one and the content process can be killed at any points.

> Note that we cannot guarantee that after an app calls commit(), it
> will never have that same system message delivered again, because the
> app might get killed while running commit().

Our current acknowledgement mechanism doesn't take this into consideration. You're pointing out another potential bug here... We need to carefully go through the system message mechanism again based on the assumption that the content process can be 1) killed or 2) at any edge-case points.

Gene

[Gene Lian]

----- Original Message -----
> From: "Gene Lian" <cl...@mozilla.com>
> To: "Justin Lebar" <justin...@gmail.com>
> Cc: "dev-webapi" <dev-w...@lists.mozilla.org>, "William Chen" <wc...@mozilla.com>, "Anne van Kesteren"
> <avanke...@mozilla.com>, "Doug Turner" <do...@mozilla.com>, "Mounir Lamouri" <mou...@mozilla.com>, "EDUARDO
> FULLEA CARRERA" <e...@tid.es>, dev-...@lists.mozilla.org, "Jonas Sicking" <jo...@sicking.cc>, "Fabrice Desre"
> <fab...@mozilla.com>, "Ken Chang" <kch...@mozilla.com>, mozilla...@lists.mozilla.org
> Sent: Tuesday, May 7, 2013 4:30:53 PM
> Subject: Re: Improving system messages to support webpages and more
>

> Our current acknowledgement mechanism doesn't take this into
> consideration. You're pointing out another potential bug here... We
> need to carefully go through the system message mechanism again
> based on the assumption that the content process can be 1) killed or
> 2) at any edge-case points.

^^ should be "2) launched". Sorry.

Gene

[Chuck Lee]

If we are going to extend the manifest for system message, it is
possible to add another syntax defining a subtype(or filter, whatever)
of system message?

This idea comes from WAP Push. It can carry various of content types[1]
including SI, SL, and CP.
If we use one system message, say "wap-push-received", to notify
application, once a WAP Push message is received, all apps registered to
"wap-push-received" will received the notification.
If an app only cares for WAP-Push-CP, it has to receive all WAP Push
messages then filter out CP messages.
I think it's unnecessary to trigger apps to receive messages they are
not interested in.
On the other hand, I don't think its a good solution to define system
messages for each content type, based on the number of types.

With the extra syntax, say "subtype", I can define single system message
and let app decide the message to receive.
It would look like

"messages": [
{ "wap-push-message": { subtype: ["application/vnd.wap.sic",
"application/vnd.wap.slc",
"application/vnd.wap.connectivity-wbxml"],
eventPage: "/wappush.html" } }
]

[1] http://www.wapforum.org/wina/wsp-content-type.htm

於 2013/5/7 上午 03:20, Jonas Sicking 提到:

[Gene Lian]

Please see my in-line comments.

----- Original Message -----
> From: "Justin Lebar" <justin...@gmail.com>
> To: "Jonas Sicking" <jo...@sicking.cc>

> Cc: "dev-webapi" <dev-w...@lists.mozilla.org>, "William Chen" <wc...@mozilla.com>, "Anne van Kesteren"
> <avanke...@mozilla.com>, "Doug Turner" <do...@mozilla.com>, "Mounir Lamouri" <mou...@mozilla.com>, "EDUARDO
> FULLEA CARRERA" <e...@tid.es>, dev-...@lists.mozilla.org
> Sent: Tuesday, May 7, 2013 9:23:03 AM
> Subject: Re: Improving system messages to support webpages and more
>

> > I agree that we need to add a commit() function. A few related
> > questions:
>
> Just to be clear, commit() alone does not solve the question of
> idempotence; I don't want us to gloss over that. We have to modify
> all of the messages (or at least their semantics) so they're safe to
> re-deliver.

We've already had an acknowledgement mechanism very similar to the role of commit(). Please see my previous comment in the same thread and let me know if they're actually different concepts.

>
> > * Should we up an apps OOM priority until it has called commit()?
> > At
> > least above other background apps.
>
> Right now, we give apps higher OOM priority when running their system
> message handlers. An app can then request higher OOM priority after
> that with a wake lock. I think this is probably fine for now.

No matter it's done by a explicit call commit() or the automatic acknowledgement internally done within the SystemMessageManager.js. I'm wondering what is the main concept of this kind of commit()? Does it imply (1) the content process has received the system message or (2) the content process has finished handling the system message? There could be a long time period between (1) and (2) depending on the content design.

Gene

[Gene Lian]

Please see the in-line comments. Thanks!

----- Original Message -----
> From: "Chuck Lee" <chu...@mozilla.com>
> To: "Jonas Sicking" <jo...@sicking.cc>
> Cc: "dev-webapi" <dev-w...@lists.mozilla.org>, "William Chen" <wc...@mozilla.com>, "Anne van Kesteren"
> <avanke...@mozilla.com>, "Doug Turner" <do...@mozilla.com>, "Mounir Lamouri" <mou...@mozilla.com>, "EDUARDO
> FULLEA CARRERA" <e...@tid.es>, dev-...@lists.mozilla.org
> Sent: Tuesday, May 7, 2013 4:50:48 PM
> Subject: Re: Improving system messages to support webpages and more
>
>

> If we are going to extend the manifest for system message, it is
> possible to add another syntax defining a subtype(or filter,
> whatever)
> of system message?

Good point. I agree with you but for a different reason. Please see the following example for registering activities in the manifest of Camera App and the previous discussion thread at [1]:

"activities": {
"record": {
"filters": {
"type": ["photos", "videos"]
},
"disposition": "window"
},
"pick": {
"filters": {
"type": ["image/*", "image/jpeg"]
},
"returnValue": true,
"disposition": "inline",
"href": "/index.html#pick"
}
},

The issue is the system message mechanism cannot distinguish system messages with different activity types because they actually share the same "activity" system message type, which means we have to queue the pending system messages by the same "activity" type even if they should belong to different queues (one for something like "activity-record" and the other for "activity-pick").

If we can support sub-types then we can also solve this issue for activities.

[1] https://groups.google.com/forum/?fromgroups=#!topic/mozilla.dev.webapi/wBVJdotpx0c

Gene

[EDUARDO FULLEA CARRERA]

On 7 may 2013 at 10:50:48, Chuck Lee wrote:
>
> If we are going to extend the manifest for system message, it is
> possible to add another syntax defining a subtype(or filter, whatever)
> of system message?
>

It makes sense to me.

An comment on your example, orthogonal to this discussion. The most relevant info to route a WAP Push message is the Application ID [1] rather than the Content Type.
Most updated version of the Content Type registry than the one you refer is available at [2]

[1] http://technical.openmobilealliance.org/Tech/omna/omna-push-app-id.aspx
[2] http://technical.openmobilealliance.org/Tech/omna/omna-wsp-content-type.aspx

________________________________

Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo.
This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at:
http://www.tid.es/ES/PAGINAS/disclaimer.aspx

[Justin Lebar]

>> We probably also need to add "commit" semantics to system messages: An app
>> would "commit" a system message after it's completely handled it, and the
>> main process could then stop re-delivering this system message.
>
> In our current design, we do support an acknowledgement mechanism like the
> "commit()" you're talking about here. That is, the content process will
> internally send an acknowledgement back to the main process to clean up the
> pending message whenever the content process successfully receives a system
> message (please search "SystemMessageManager:Message:Return:OK" in
> SystemMessageManager.js).

The acknowledgement we have is not the same as the commit I'm suggesting we
need.

The current acknowledgement is a system-level ack that's sent as soon as the
system message handler function completes.

But that does not necessarily mean that the app has handled the system message.
For example, an app might want to write to IndexedDB as a result of the system
message. The app would kick off the IndexedDB write, and at that point we
would send the existing ack. But from the application's perspective, the
message has not yet been committed, and if the app gets killed during the
IndexedDB write, it must receive the message again.

What I think we need is an /application-level/ ack (or you could think
of it as an "end-to-end" [1] ack), not a system-level ack. An
application-level ack would indicate that the application has fully
handled the message and no longer needs it re-delivered if it crashes.

-Justin

[1] http://en.wikipedia.org/wiki/End-to-end_principle

On Tue, May 7, 2013 at 4:30 AM, Gene Lian <cl...@mozilla.com> wrote:
> Please see my in-line comments. Looping the b2g mailing list since some of the discussions might not be highly related to the point of view of Web APIs.
>

> ----- Original Message -----
>> From: "Justin Lebar" <justin...@gmail.com>
>> To: "Jonas Sicking" <jo...@sicking.cc>
>> Cc: "dev-webapi" <dev-w...@lists.mozilla.org>, "William Chen" <wc...@mozilla.com>, "Anne van Kesteren"
>> <avanke...@mozilla.com>, "Doug Turner" <do...@mozilla.com>, "Mounir Lamouri" <mou...@mozilla.com>, "EDUARDO
>> FULLEA CARRERA" <e...@tid.es>, dev-...@lists.mozilla.org
>> Sent: Tuesday, May 7, 2013 3:57:48 AM
>> Subject: Re: Improving system messages to support webpages and more
>>

>> We probably also need to add "commit" semantics to system messages:
>> An
>> app would "commit" a system message after it's completely handled it,
>> and the main process could then stop re-delivering this system
>> message.
>
> In our current design, we do support an acknowledgement mechanism like the "commit()" you're talking about here. That is, the content process will internally send an acknowledgement back to the main process to clean up the pending message whenever the content process successfully receives a system message (please search "SystemMessageManager:Message:Return:OK" in SystemMessageManager.js). Let me try to elaborate more regarding our system message behaviours as below:
>
> When the main process attempts to fire system message, it will do the following 3 things:
>
> 1. Deliver the system message to the app (if it's alive).
> 2. Keep a copy of the system message in the pending queue (no matter the app is alive or not).
> 3. Try to launch the app (no matter it's alive or not).
>
> So we can discuss the following two scenarios before delivering the system message:
>
> A. The app (content process) is alive:
>
> The main process knows the app is running (at least at that moment) so it fires system message to the target content process (step #1). When the content process receives the system message, it will immediately fire an acknowledgement back to the main process. The main process will then clean up the pending message (queued by step #2) because it has been successfully delivered for sure. Note that the attempt of opening the app (step #3) doesn't really help because the app has already been running. However, we could still have an edge case that the app can probably be killed (due to OOM'ed) before receiving the system message or sending the acknowledgement back. Under this circumstance, the step #3 will eventually take effect and launch the app, then it will go to the following case B.
>
> B. The app (content process) is NOT alive:
>
> Obviously, the main process won't fire the system message to the content process because the main process understands the target isn't alive. Anyway, the main process will keep the system message in the pending queue (step #2). Later, the step #3 will then launch the app in the long run. When the launching app calls .mozSetMessageHandler(...), it will try to retrieve all the pending system messages from the central and clean them up at the same time [1]. If the app is launched manually by users instead of being launched by step #3, everything still goes the same except that the step #3 doesn't take effect.
>
> [1] A potential bug comes into my mind that we should also fire an acknowledgement back to clean up the system message, instead of cleaning it up during the retrieving process, because the retrieving process is an async one and the content process can be killed at any points.
>
>> Note that we cannot guarantee that after an app calls commit(), it
>> will never have that same system message delivered again, because the
>> app might get killed while running commit().
>

> Our current acknowledgement mechanism doesn't take this into consideration. You're pointing out another potential bug here... We need to carefully go through the system message mechanism again based on the assumption that the content process can be 1) killed or 2) at any edge-case points.
>

> Gene

[Anne van Kesteren]

On Mon, May 6, 2013 at 12:20 PM, Jonas Sicking <jo...@sicking.cc> wrote:
> Thoughts?

I think long term this should be standardized as part of the
controller thingie:
https://github.com/slightlyoff/NavigationController/ I don't really
have opinions on what we do meanwhile I think.


--
http://annevankesteren.nl/

[Jonas Sicking]

So how do we make that happen? Proposals welcome. I don't want to
build something we know we need to throw away.

/ Jonas

[Anne van Kesteren]

On Tue, May 7, 2013 at 10:03 AM, Jonas Sicking <jo...@sicking.cc> wrote:
> So how do we make that happen? Proposals welcome. I don't want to
> build something we know we need to throw away.

My idea is that there's some way within that worker to register for
push notifications and end-user notifications which can spin up the
worker if inactive.

The only problem is that these workers have origin + URL scope so we
need some way to map those concepts together. Maybe the worker should
have some kind of identifier as well which you can use to associate
push/end-user notifications with them.


--
http://annevankesteren.nl/

[Jonas Sicking]

Also, do we want to force people to install a navigation controller
just so that they can handle clicks on WebNotifications that happen
after the window is closed?

/ Jonas

[Jonas Sicking]

Sure, but at least the API will remain stable there once we move them
to a worker. I.e. we did build a throw-away implementation, but not a
throw-away API which we'd have to fight to move people off of.

/ Jonas

On Tue, May 7, 2013 at 10:06 AM, Doug Turner <do...@mozilla.com> wrote:
> We did that for the Social API, didn't we? Creating iframes on the hidden
> window to simulate background workers.
>
>
> Jonas Sicking wrote:

>>
>> On Tue, May 7, 2013 at 9:29 AM, Anne van Kesteren<ann...@annevk.nl>
>> wrote:
>>>
>>> On Mon, May 6, 2013 at 12:20 PM, Jonas Sicking<jo...@sicking.cc> wrote:
>>>>
>>>> Thoughts?
>>>
>>> I think long term this should be standardized as part of the
>>> controller thingie:
>>> https://github.com/slightlyoff/NavigationController/ I don't really
>>> have opinions on what we do meanwhile I think.
>>
>>

>> So how do we make that happen? Proposals welcome. I don't want to
>> build something we know we need to throw away.
>>

>> / Jonas

[Anne van Kesteren]

On Tue, May 7, 2013 at 10:51 AM, Jonas Sicking <jo...@sicking.cc> wrote:
> Also, do we want to force people to install a navigation controller
> just so that they can handle clicks on WebNotifications that happen
> after the window is closed?

I think for push notifications it's an acceptable requirement given
the complexity of push notifications in general. For end-user
notifications we might something like what I proposed in
http://lists.w3.org/Archives/Public/public-whatwg-archive/2013Mar/0282.html
I think.

Either way we're running in the problem of what's considered part of
the application and what's not. If applications were origin-scoped the
answers are easy, but since they're not...


--
http://annevankesteren.nl/

[Jonas Sicking]

On Tue, May 7, 2013 at 11:09 AM, Anne van Kesteren <ann...@annevk.nl> wrote:
> On Tue, May 7, 2013 at 10:51 AM, Jonas Sicking <jo...@sicking.cc> wrote:
>> Also, do we want to force people to install a navigation controller
>> just so that they can handle clicks on WebNotifications that happen
>> after the window is closed?
>
> I think for push notifications it's an acceptable requirement given
> the complexity of push notifications in general. For end-user
> notifications we might something like what I proposed in
> http://lists.w3.org/Archives/Public/public-whatwg-archive/2013Mar/0282.html
> I think.

It's a bit scary to rely on that people will be able to register the
click message fast enough. In particular, if you nest Future callbacks
a couple of levels, for example by using Future.every to wait for
multiple things, you can easily miss it.

> Either way we're running in the problem of what's considered part of
> the application and what's not. If applications were origin-scoped the
> answers are easy, but since they're not...

Indeed.

/ Jonas

[Salvador de la Puente González]

Hello

I totally agree with Justin. Just a suggestion from the point of view of
a developer:

In order to keep the compatibility with current software or keep code
modification as low as possible, can we rely on a new parameter to the
manifest saying we need the application commit that Justin is suggesting?

Something like:

"messages": [
{ "sms-sent": "/message_handler.html", "auto-commit": "false" },
// This message need to send app-ack
{ "telephony-call-ended": "/message_handler.html" }, // This
message is auto-ack by the sys level ack
]

This way, current software is "auto-commited" but critical handling can
turn this flag off to "manually" commit.

What do you think?

[Mounir Lamouri]

On 07/05/13 09:50, Chuck Lee wrote:
>
> If we are going to extend the manifest for system message, it is
> possible to add another syntax defining a subtype(or filter, whatever)
> of system message?
>

> This idea comes from WAP Push. It can carry various of content types[1]
> including SI, SL, and CP.
> If we use one system message, say "wap-push-received", to notify
> application, once a WAP Push message is received, all apps registered to
> "wap-push-received" will received the notification.
> If an app only cares for WAP-Push-CP, it has to receive all WAP Push
> messages then filter out CP messages.
> I think it's unnecessary to trigger apps to receive messages they are
> not interested in.
> On the other hand, I don't think its a good solution to define system
> messages for each content type, based on the number of types.
>
> With the extra syntax, say "subtype", I can define single system message
> and let app decide the message to receive.
> It would look like
>
> "messages": [
> { "wap-push-message": { subtype: ["application/vnd.wap.sic",
> "application/vnd.wap.slc",
>
> "application/vnd.wap.connectivity-wbxml"],
> eventPage: "/wappush.html" } }
> ]

Unfortunately, 'subtype' means nothing for system messages because the
passed object has a 'type' attribute but that is the only known
attribute to the UA. We can't add a 'subtype' attribute given that some
messages will not have a 'subtype' at all. The solution for this would
be to have some kind of filters like Web Activities have but this is
terribly complex and unfriendly.

In my opinion, the best way to solve that problem is to send the message
to an event page or a worker and from there open the page you want
depending on the message type. Why wouldn't that work?

Thanks,
--
Mounir

[Mounir Lamouri]

On 06/05/13 23:56, Jonas Sicking wrote:
> On Mon, May 6, 2013 at 12:57 PM, Justin Lebar <justin...@gmail.com> wrote:
>> While we're modifying system messages, another issue we need to
>> consider is how to deal with apps which get killed while handling a
>> system message.
>>
>> In general, an app can be killed at any point in time. This is
>> particularly true for apps which handle system messages, since they
>> tend to run in the background, and so can be OOMed by either the main
>> process or the foreground app.
>>
>> The only way I'm aware of to handle this properly is to make system
>> messages idempotent. That is, we must change the semantics of system
>> messages such that if an app receives a system message twice, that has
>> the same effect as receiving it once.
>>

>> We probably also need to add "commit" semantics to system messages: An
>> app would "commit" a system message after it's completely handled it,
>> and the main process could then stop re-delivering this system
>> message.
>>

>> Note that we cannot guarantee that after an app calls commit(), it
>> will never have that same system message delivered again, because the

>> app might get killed while running commit(). In general, we suffer
>> from the Byzantine Generals problem here, so from the app's
>> perspective, commit() is necessary but not sufficient to stop
>> receiving a given message.
>
> Yes. All good points.
>

> I agree that we need to add a commit() function.

In most situations, I suppose that keeping the message alive until we
leave the handler would just work. I guess we might find extreme cases
where even in a non memory constrained environment we need to keep the
message alive longer. I guess some applications might be sensitive
enough (like dialer) that they don't want to lose a message.

Given that I guess these situations are going to be the minority, we
could probably simply discard the message when the handler is left (and
the application is still alive at that point) unless .delay() has been
called on the message. In that case, we keep the message alive until the
application calls .pop() or whatever method.

However, this creates the issue of time out: it would be quite hard to
define a time out value for the application given that we might end up
in a race condition. A solution could be to pass a Future to .delay() so
the message would be discarded when the Future would be resolved (or
rejected). It doesn't solve much the time out issue but reduces the
chances that the developer simply never calls .pop().

How does that sound?

--
Mounir

[Mounir Lamouri]

On 07/05/13 17:29, Anne van Kesteren wrote:
> On Mon, May 6, 2013 at 12:20 PM, Jonas Sicking <jo...@sicking.cc> wrote:
>> Thoughts?
>
> I think long term this should be standardized as part of the
> controller thingie:
> https://github.com/slightlyoff/NavigationController/ I don't really
> have opinions on what we do meanwhile I think.

Isn't the purpose of the Network/Navigation Controller to intercept all
network requests and handle them a way or another? I'm afraid I might be
missing something :(

--
Mounir

[Mounir Lamouri]

On 06/05/13 20:20, Jonas Sicking wrote:
[...]
> "messages": [
> { "alarm": { page: "/index.html" } }
> ]
>
[...]

> We can then add support for
>
> "messages": [
> { "alarm": { eventPage: "/event-page.html" } }
> ]
>

[...]

> We could also eventually add support for something like:
>
> "messages": [
> { "alarm": { worker: "/messageHandler.js" } }
> ]

What about:
{
"alarm": {
"href": "/index.html",
"background": false
}
}

{
"alarm": {
"href": "/event-page.js",
"background": true
}
}

{
"alarm": {
"href": "/messageHandler.js",
"background": true,
"worker": true
}
}

"worker" would be false by default.
I don't know what should be "background" default value.

Also, I believe that we should only take a js file in any case unless we
expect people to have some HTML in their event-page? We could simply
inject the js file in an HTML document. Is there a reason why Chrome
uses an HTML file instead of a plain JS file?

> In order to support system messages for web pages we can then borrow
> from this syntax. So for example registering for a push notification
> from a web page could look like:
>
> navigator.push.register({ page: "/foo.html"});
>
> or
>
> navigator.push.register({ eventPage: "/foo.html"});

What would be the benefit of using navigator.push.register() compared to:

navigator.foo.register("push", { href: "/foo.js", background: true });
(With "foo" being a placeholder name for "messages" or "systemMessages"
or whatever.)

Being able to register and unregister system messages is something web
applications will be looking for. Actually, IIRC, there is an open bug
for this in the runtime specification.

Cheers,
--
Mounir

[Mounir Lamouri]

FWIW, Chrome doesn't seem to enforce using HTML files for Event Pages.
It is actually JS only for those:
http://developer.chrome.com/extensions/event_pages.html

However, JS and HTML are allowed for background pages but it isn't
really clear what the HTML pages would provide in addition of simply
have some markup ready:
http://developer.chrome.com/extensions/background_pages.html

--
Mounir

[Salvador de la Puente González]

Hello

IMHO, I think all system-messages should be always attended by a pure JS
handler (a worker, for instance). I want to mean: with **no interface**
and **in background** so receiving a message never changes the user
interface directly. The handler could receive an array with the windows
in the same domain currently opened (at most one in FirefoxOS or several
tabs / windows in Firefox) in order to interact with them and, as
suggested in the thread, it can always use window.open() to force
another view to be shown.

This way the manifest is not altered at all.

For web-pages, where we don't have manifest, we can add this section
dynamically, for example:

navigator.manifest.update('messages', [{"alarm": "url/to/messageHandler.js"}])


What are your thoughts about this?

Thank you.

[Gene Lian]

Hi Justin, please see my in-line response. Thanks!

----- Original Message -----
> From: "Justin Lebar" <justin...@gmail.com>
> To: "Gene Lian" <cl...@mozilla.com>
> Cc: "dev-webapi" <dev-w...@lists.mozilla.org>, "<dev-...@lists.mozilla.org>" <dev-...@lists.mozilla.org>
> Sent: 2013年5月7日 星期二 23:56:58
> Subject: Re: Improving system messages to support webpages and more
>

> >> We probably also need to add "commit" semantics to system
> >> messages: An app
> >> would "commit" a system message after it's completely handled it,
> >> and the
> >> main process could then stop re-delivering this system message.
> >

Good point! One separate point I'm curious about is: if the app doesn't call commit() for a long time, how can the central know the app has already crashed or it's actually busy with its tasks? Does a timeout mechanism make sense here? Or should the central need an active way to frequently ask whether an app is still alive or not if it's having some pending system messages?

Gene

[Anne van Kesteren]

The idea is for it to also address these use cases. Having another
worker/page for that would be wasteful.


--
http://annevankesteren.nl/

[Chuck Lee]

於 2013/5/8 下午 06:38, Mounir Lamouri 提到:

I like to make 'subtype' optional, so all system message will be
received if 'subtype' not defined.

If we add new syntax to system message, the are acting like filters too.
I think 'subtype' should only add a check loop in
|SystemMessageInternal._isPageMatched()|

Even if it extends to a general filter for system message data, which
just pops into my mind, like

"messages": [
{ "wap-push-message": { filter: [ {contentType: "application/vnd.wap.sic"},
{contentType: "application/vnd.wap.slc"}],
...
}
}
]

to accept only system messages satisfies
|message.contentType === "application/vnd.wap.sic" ||
message.contentType === "application/vnd.wap.slc"|

I think it's not yet reach the "terribly complex" level, but might be
slow if there is a large filter.

For the unfriendly issue, my opinion is, for designers writing apps to
receive certain system message, they should know the data structure in
the system message.
Since they know the message structure, it's not difficult for them to
write a such filter in manifest.
And once they put the filter in manifest, they don't have to check and
pick messages they want from every incoming system messages.
I think it's friendly.

> In my opinion, the best way to solve that problem is to send the message
> to an event page or a worker and from there open the page you want
> depending on the message type. Why wouldn't that work?

Adding 'subtype' is for the condition when several apps are listening to
same system message, each app cares about different 'type'.
In such case, a worker in one app can't handle/dispatch message for
other apps, or it is acting like a system-message-like-service in
content process.
I think it shouldn't even it can.
> Thanks,

[Mounir Lamouri]

So then, shouldn't we figure out how to have a generic background
worker/page and see how to plug stuff into it instead of plugging
everything into a Network/Navigation thingy?

--
Mounir

[Mounir Lamouri]

On 09/05/13 04:36, Chuck Lee wrote:
> "messages": [
> { "wap-push-message": { filter: [ {contentType:
> "application/vnd.wap.sic"},
> {contentType:
> "application/vnd.wap.slc"}],
> ...
> }
> }
> ]
>

> I think it's not yet reach the "terribly complex" level, but might be
> slow if there is a large filter.

Sure. But then, what do you do for things that have optional properties,
should you consider the missing property as not fulfilling the
condition? What about people trying to do "application/*"?
Also, I never said it would be slow but it would be complex. This is
unlikely going to be slow but you can quickly reach a situation where
understanding what is happening, why and how is not trivial. A good API
is straightforward to understand.

>> In my opinion, the best way to solve that problem is to send the message
>> to an event page or a worker and from there open the page you want
>> depending on the message type. Why wouldn't that work?
> Adding 'subtype' is for the condition when several apps are listening to
> same system message, each app cares about different 'type'.
> In such case, a worker in one app can't handle/dispatch message for
> other apps, or it is acting like a system-message-like-service in
> content process.
> I think it shouldn't even it can.

I think there is a misconception, For the moment, I do not think it is a
common situation to receive a system message you do not care about. This
situation should be avoided because a system message implies to start
the application if it was stopped so we shouldn't do that for no good
reason.

For example, two common usages of system messages are Web Activities and
Alarms. If an application is started to handle an activity, we do not
expect the application to simply ignore the request. We also do not
expect an application to not behave when it is started again by an alarm
it has self itself.

In my opinion, if you are designing a system message that is most of the
time going to start applications that didn't care about it, you should
create multiple system messages instead of one. I do not know anything
about wap push messages (I do not even know what that is) so I
unfortunately can't help you with that design but you might want to
consider creating thing like "wap-push-sic" and "wap-push-slc".

Cheers,
--
Mounir

[Anne van Kesteren]

On Thu, May 9, 2013 at 3:46 AM, Mounir Lamouri <mou...@lamouri.fr> wrote:
> So then, shouldn't we figure out how to have a generic background
> worker/page and see how to plug stuff into it instead of plugging
> everything into a Network/Navigation thingy?

It's the controller thingy, really, so I think that's what it is.


--
http://annevankesteren.nl/

[Mounir Lamouri]

I do not see why a Network Controller is the best tool to base other
background workers on.

--
Mounir

[Thinker K.F. Li]

Since there are a lot details on this thread, I don't talk about every
item. My opinion is that we don't need a "commit" function to make sure
content code finish its work since we already have localstorage and
cookie API running in sync. Once the JS function handling system
messages is returned, we can just assume it already finishes it's work.

As you had mentioned, there some async tasks maybe lost for crash or
OOM, the content can remember the submitted system message in
localstorage or cookie before returning from the handler function and
checking it in next running. If the process is crashed before the event
have been remembered in localstorage, it means the system message will
be sent again.

--
Sinker
--
天教懶漫帶疏狂

[Tim Chien]

>From a web dev's perspective, I disagree what Thinker said there.

We discourage any usage of sync APIs that involves disk I/O. Both
localStorage and document.cookie fall into this category. Also, if web
devs should always prevent any data loss themselves by using another
API, people would probably ended up writing a abstraction library to
wrap it. That's not a successful API design.

That being said, I am not sure if our system message usage always
involves saving the message to the disk. Still, as save-to-disk use
case exists, we would still need that |commit()| method; for non-async
operation, people can |commit()| directly in the callback function.

On Mon, May 13, 2013 at 7:09 PM, Thinker K.F. Li <thi...@codemud.net> wrote:
> Since there are a lot details on this thread, I don't talk about every
> item. My opinion is that we don't need a "commit" function to make sure
> content code finish its work since we already have localstorage and
> cookie API running in sync. Once the JS function handling system
> messages is returned, we can just assume it already finishes it's work.
>
> As you had mentioned, there some async tasks maybe lost for crash or
> OOM, the content can remember the submitted system message in
> localstorage or cookie before returning from the handler function and
> checking it in next running. If the process is crashed before the event
> have been remembered in localstorage, it means the system message will
> be sent again.
>
> Justin Lebar <justin...@gmail.com> writes:
>

> --
> Sinker
> --
> 天教懶漫帶疏狂

> _______________________________________________
> dev-webapi mailing list
> dev-w...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-webapi

--
Tim Guan-tin Chien, Engineering Manager and Front-end Lead, Firefox
OS, Mozilla Corp. (Taiwan)

[Mounir Lamouri]

I'm not really sure what you are replying to but generally speaking the
Social API isnt't really an API as it is not exposed to real Web
content. Only some whitelisted partners have access to it. Therefore,
problems that apply to Web APIs might not apply to the Social feature in
Firefox such as changing the API and its behaviour.

--
Mounir

On 07/05/13 18:06, Doug Turner wrote:
> We did that for the Social API, didn't we? Creating iframes on the
> hidden window to simulate background workers.
>
> Jonas Sicking wrote:

>> On Tue, May 7, 2013 at 9:29 AM, Anne van Kesteren<ann...@annevk.nl>

>> wrote:
>>> On Mon, May 6, 2013 at 12:20 PM, Jonas Sicking<jo...@sicking.cc> wrote:
>>>> Thoughts?

>>> I think long term this should be standardized as part of the
>>> controller thingie:
>>> https://github.com/slightlyoff/NavigationController/ I don't really
>>> have opinions on what we do meanwhile I think.
>>

>> So how do we make that happen? Proposals welcome. I don't want to
>> build something we know we need to throw away.
>>
>> / Jonas

[nsm.n...@gmail.com]

On Monday, May 6, 2013 12:20:05 PM UTC-7, Jonas Sicking wrote:
> Hi All,
>
>
>
> We're currently struggling a bit with a couple of things for system
>
> messages. For background on system messages, see the thread here
>
>
>
> https://groups.google.com/forum/#!topic/mozilla.dev.webapi/o8bkwx0EtmM/discussion
>
>
>
> However the current implementation of system messages only works well
>
> for apps where either
>
> * The app consists of only a single webpage.
>
> * The app wants to always navigate the user to a specific page
>
> whenever there's an incoming message of the given type.
>
>
>
> In other cases the app risks navigating the user as soon as there's an
>
> incoming message, whether handling that message needs UI or not.
>
>
>
> Also, any time there's an incoming message, even if the app starts in
>
> the background, the app shows up in the application switcher. This can
>
> be confusing to users since they never started the app.
>
>
>
> We also have the problem that since system messages rely on manifests,
>
> they only work for apps currently, not for normal web pages. While
>
> we're planning on making it possible for websites to create a manifest
>
> for their web pages, forcing people to create a manifest just because
>
> they want to handle a system message.
>
>


Just adding on to Jonas' comment to reduce dependencies on manifest files.

I don't think the current system messages API (where the web API and system messages both track the manifest URL) is a good idea. It is a bit magical and assumes this clear 'app'-ness that isn't a good idea for the web. Rather WebAPIs should focus on pages, and that the page is inside an app should be incidental as far as possible. This fits in with the web model of permissions being tied to origin and not to manifests. Bug 800431 [1] clearly highlights developer confusion arising due to reliance on manifests and moving the handler declaration away from the actual call which triggers the handler. Lets instead shift responsibility of identifying the correct handler page to the WebAPI and away from the manifest. That is, in the future the Alarms API would be

navigator.mozAlarms.add(new Date(), "ignoreTimezone", { data: 'foo-wakemeup', eventPage: '/awesomeEventHandler.js' })

where all the optional parameters have been grouped into the object literal. If no eventPage/page is specified, the current page (index.html) becomes the system message receiver as if it was called like this:

navigator.mozAlarms.add(new Date(), "ignoreTimezone", { data: 'foo-wakemeup', page: '/index.html' })

In both cases the "messages" entries in the manifest are completely ignored. The eventPage/page URLs are converted to absolute URIs based on the origin of the caller of navigator.mozAlarms.add(). The mozAlarms page then internally calls SystemMessageInternal.registerPage() or similar with the relevant message name and URI. Most APIs require these message registrations to persist after Web Runtime restart. Since the web APIs are required to store the message <-> web page association, it makes sense to delegate responsibility of re-registering every web page for the right system message at boot.

The way I see it, there are two kinds of system messages, one where the message is unique, specific and is explicitly added by the page, and the other where the page expresses interest in the message but does not drive/activate it.

1) mozAlarms and SimplePush are both examples of this. Here the web developer creates a unique entry for a *specific* alarm on push notification and wishes to receive a system message only when that specific entry triggers. In this case the *manifest is completely irrelevant* even when the page is part of an app. Instead the above forms of the APIs are used. The WebAPI is in charge of calling SMI.registerPage(). It will have to do this even when the phone reboots and restore any entries, because the manifest should not enforce these or the manifest may not exist (in case of an call from a webpage). The WebAPI implementation is also in charge of *prompting for permission* since system messages usually require a certain permission. When the same page was run as part of an app, this permission could be offloaded to the manifest, so the WebAPI should check for this and not prompt unnecessarily.

2) Bluetooth or Battery API is a good example of the second kind, where the page wants to know when a bluetooth device is attached, but cannot control this. These messages are sent by the web runtime (generally using broadcastMessage). Now imagine a developer creates a web *page* that monitors for battery and when it goes below 10% frantically plays loud music and vibrates to notify the user that it is critical to charge the phone. Since the web page has no manifest, the battery API would have to be modified to something like:

navigator.battery.addMessageListener('levelchanged', { eventPage: '/allHellBreaksLoose.js' })

The catch with this being a page is that until the user visits the page at least once, the eventPage will not be loaded when levelchanged happens. Which is fine because that is what users expect from web pages.

But when the webpage is installed as part of an *app*, the manifest would declare the same message handler, and the system messages API would use this, as happens right now. [2]

Coming to the implementation of eventPage. The way the social API works is by emulating a shared worker, and it does indeed accept a JS file and not an HTML file. The JS file is run in the context of an hidden iframe within the same origin, with the actual script evaluation performed in Components.utils.Sandbox so that extensions to the window object can be made relatively safely. My plan is to add a window.launch(URI, data) similar to window.open() so that this hidden eventPage can launch UI. The behaviour of window.launch() will be:

* The URI has to be in the same origin as the eventPage (which is in the same origin as the page that called the WebAPI).
* In the browser, the browser will check if such a tab is already open. If such a tab is found skip this step, otherwise launch a new tab with the URI.
* Send the tab a system message 'launched' or similar with the data parameter.

Right now there is no communication channel from the eventPage to the newly launched window, apart from the system message. I would also like to add a API so the eventPage could see if any existing tabs are open from the same origin, and send them a system message instead.

On FxOS, the call to launch() will launch the page in the browser, or launch the app if the page is part of an app.

Would it make sense to not have the launch() call and instead force the page to launch a desktop-notification, whose onclick() would then open a tab?

I would like thoughts on this approach.

Regards,
Nikhil

[1] https://bugzil.la/800431
[2] Incidentally, the battery API should probably be changed to use system messages rather than event handlers, since those can run even when the app/webpage is not running.