DataStore security model (was: Re: Gaia wide "save" discussion.)

[Mounir Lamouri]

On 04/05/13 14:27, Paul Theriault wrote:
> Have you thought about the security model for DataStores yet? On the wiki there isn't any information about what requirements an app would have to satisfy to access a datastore. Or is the idea that these would be completely public (i.e. any app could read&write to any store, so long as they declared this intention in their manifest)

All DataStore would be read-only by default. For a v1, we want to stop
there because read-write might be complex in the sense of an application
might allow app A to write but not app B.

When an application wants to read a store, the user will be prompted,
asking for permission to share the information.

This is the basic security model we have for the moment. I forwarded the
discussion to dev-webapi because this topic is more appropriate there.

Cheers,
--
Mounir

[Jonas Sicking]

What will the text in the prompts say for DataStores that are of types
that the system doesn't understand. I.e. for a "contacts" prompt we
can easily put a useful description in the prompt since we know what
type of data "contacts" contains. However what would the security
dialog for a store of type "favoritefriends" exposed by a social
networking app look like?

Also, what happens if there are 3 apps that expose a "favoritefriends"
store? Will the user receive 3 separate dialogs once an app tries to
read from "favoritefriends"?

/ Jonas

[Mounir Lamouri]

On 14/05/13 06:39, Jonas Sicking wrote:
> On Tue, May 7, 2013 at 9:47 AM, Mounir Lamouri <mou...@lamouri.fr> wrote:
>> On 04/05/13 14:27, Paul Theriault wrote:
>>> Have you thought about the security model for DataStores yet? On the wiki there isn't any information about what requirements an app would have to satisfy to access a datastore. Or is the idea that these would be completely public (i.e. any app could read&write to any store, so long as they declared this intention in their manifest)
>>
>> All DataStore would be read-only by default. For a v1, we want to stop
>> there because read-write might be complex in the sense of an application
>> might allow app A to write but not app B.
>>
>> When an application wants to read a store, the user will be prompted,
>> asking for permission to share the information.
>>
>> This is the basic security model we have for the moment. I forwarded the
>> discussion to dev-webapi because this topic is more appropriate there.
>
> What will the text in the prompts say for DataStores that are of types
> that the system doesn't understand. I.e. for a "contacts" prompt we
> can easily put a useful description in the prompt since we know what
> type of data "contacts" contains. However what would the security
> dialog for a store of type "favoritefriends" exposed by a social
> networking app look like?

We have a "description" field in DatStore declaration, don't we?

> Also, what happens if there are 3 apps that expose a "favoritefriends"
> store? Will the user receive 3 separate dialogs once an app tries to
> read from "favoritefriends"?

We can use a checkbox list but then we need to write something generic
enough. It could be:
"""
This application wants to access data from the following applications.
You can check the data stores you want this application to access:
[ ] Foo
[ ] Bar
[ ] Foobar
"""

I'm not saying that those solutions are perfect but worth trying.

--
Mounir

[Thinker K.F. Li]

If the user deny the access for some of all providers, will the user see
the prompt everytime unless all are agreed? Or, we should provide a way
that applications can request for authorizing for one or several
providers later.

--
Sinker
--
天教懶漫帶疏狂

[Mounir Lamouri]

On 15/05/13 10:26, Thinker K.F. Li wrote:

> If the user deny the access for some of all providers, will the user see
> the prompt everytime unless all are agreed? Or, we should provide a way
> that applications can request for authorizing for one or several
> providers later.

I do not think we should provide a way for applications to select some
providers but we should probably find a UI that will allow users to
change their mind regarding sharing.

We are starting the implementation of the API and Gaia will be involved
to do some experimentations so hopefully our frontend team will have a
solution for those UI issues.

--
Mounir

[Andrew Sutherland]

On 05/15/2013 07:11 AM, Mounir Lamouri wrote:
> We are starting the implementation of the API and Gaia will be involved
> to do some experimentations so hopefully our frontend team will have a
> solution for those UI issues.

All of the examples for the DataStore API
(https://wiki.mozilla.org/WebAPI/DataStore) that I've seen seem pretty
specific to the contacts use-case where the upstream source of the
information has a terms of service that means we aren't allowed to
freely integrate the data with the local contacts DB. For example, Facebook.

It seems like it would be better to prioritize the more-generic "Inter
App Communication" API
(https://wiki.mozilla.org/WebAPI/Inter_App_Communication) and let the
Contacts app evolve a protocol before trying to standardize it since
there is just the one planned user.

This would also benefit other apps like e-mail for which the DataStore
is insufficient but the ability to expose data to another app would
allow a neat class of apps to flourish. Especially if the
inter-app-communication API allowed it to place strong requirements on
other apps that might access it. For example, if the other app could be
required to be 100% sandboxed so that it can only access its
locally-stored resources defined to be part of the app and make no
network requests, concerns about information leakage could be largely
side-stepped. (Presumably more useful apps would need the ability to
trigger a web activity, which could be a means of surreptitious
information egress, but would at least require user to broadly approve
the action.)

Andrew

[Mounir Lamouri]

On 16/05/13 20:19, Andrew Sutherland wrote:
> On 05/15/2013 07:11 AM, Mounir Lamouri wrote:
>> We are starting the implementation of the API and Gaia will be involved
>> to do some experimentations so hopefully our frontend team will have a
>> solution for those UI issues.
>
> All of the examples for the DataStore API
> (https://wiki.mozilla.org/WebAPI/DataStore) that I've seen seem pretty
> specific to the contacts use-case where the upstream source of the
> information has a terms of service that means we aren't allowed to
> freely integrate the data with the local contacts DB. For example,
> Facebook.

Contacts is one use case but there are other. We would like to use that
for messaging too: SMS and MMS such as other kind of messages could be
stored in a "Message" DataStore.

> It seems like it would be better to prioritize the more-generic "Inter
> App Communication" API
> (https://wiki.mozilla.org/WebAPI/Inter_App_Communication) and let the
> Contacts app evolve a protocol before trying to standardize it since
> there is just the one planned user.
>
> This would also benefit other apps like e-mail for which the DataStore
> is insufficient but the ability to expose data to another app would
> allow a neat class of apps to flourish. Especially if the
> inter-app-communication API allowed it to place strong requirements on
> other apps that might access it. For example, if the other app could be
> required to be 100% sandboxed so that it can only access its
> locally-stored resources defined to be part of the app and make no
> network requests, concerns about information leakage could be largely
> side-stepped. (Presumably more useful apps would need the ability to
> trigger a web activity, which could be a means of surreptitious
> information egress, but would at least require user to broadly approve
> the action.)

As you pointed, inter app communication would require a lot of security
considerations. Way more than DataStore. If the email app wants to share
emails to any other application, there should be a way to prevent some
applications to read the user's emails. Using origin filtering isn't
good enough because we lose the universal communication aspect of that
API. We haven't found a better system yet.

Sandboxing with rejecting network connection would be hard to do because
we would need to also reject any Web Activity and forbid the application
to open an iframe or using navigator.location. Such sandboxing wouldn't
be very web-y and would only work in a packaged application context.

Ideally, we should also start to experiment the inter app communication
api but we are unfortunately lacking man power for the moment :(

--
Mounir

[Andrew Sutherland]

On 05/17/2013 07:38 AM, Mounir Lamouri wrote:
> Contacts is one use case but there are other. We would like to use that
> for messaging too: SMS and MMS such as other kind of messages could be
> stored in a "Message" DataStore.

But we already have mozMobileMessage and it exposes a richer data
structure than the DataStore API is capable of representing (threads),
and supports richer querying (via nsIDOMMozSmsFilter) that seems more
flexible.

> As you pointed, inter app communication would require a lot of security
> considerations. Way more than DataStore. If the email app wants to share
> emails to any other application, there should be a way to prevent some
> applications to read the user's emails. Using origin filtering isn't
> good enough because we lose the universal communication aspect of that
> API. We haven't found a better system yet.

I don't understand why the security model for DataStore doesn't work
equally well for Inter App Communication, at least for an introductory
approach:

If the e-mail app defines an endpoint (pardon the use of bullet points
to approximate JSON to avoid font issues):
* "inter-app-endpoint-owned": {
** "email": {
*** "name": { "en": "Full e-mail access" },
*** "access-description": { "en": "Allow the app to access to all of
your e-mails. It will be able to read your e-mails, mark messages
read/unread, delete messages, forward messages, everything. Only allow
this for apps that have a reason to need to access your messages in this
way." }}}

And in say, an inbox-zero enabling app, optimized for helping you triage
your e-mail:
* "inter-app-endpoint-used": {
** "email": {
*** "reason": { "en": "In order to triage your e-mail, the app needs to
be able to access your inbox, as well as create folders and move
messages to those folders. }}}


Another alternative would be to do more of an oauth-style authorization
using web activities. The inbox zero app would indicate it would like
to establish authorization for a persistent connection to the email
app. Say a web activity of "inter-app-authorize", with a "type" of
"email", which the e-mail app authorizes. The requesting app could
specify very granular privileges that it desires as part of its payload
(see only the inbox, be able to delete message or 'archive' them using
existing archive rules, say), and the e-mail app could then explain
exactly what the requesting app is asking for. The user decides to
authorize, or not.

If authorized, the e-mail app does something to authorize contact.
Perhaps it calls an API call explicitly authorizing the origin in
question to contact it via message-port. This then allows the inbox
zero app to establish a message port with the e-mail app whenever it
wants, triggering the e-mail app to startup, possibly in a lightweight
background-page kind of way.

> Sandboxing with rejecting network connection would be hard to do because
> we would need to also reject any Web Activity and forbid the application
> to open an iframe or using navigator.location. Such sandboxing wouldn't
> be very web-y and would only work in a packaged application context.

Yes, this would be a new thing that's enabled by packaged applications
(or a strict interpretation of app-cache that nets out to the same
thing.) But I think it would be a great thing to be able to have fully
sandboxed apps in general. I don't need my calculator to be able to
talk to the network, nor do the tower defense games I buy and play on my
tablet. (Although they frequently want to for business reasons.) But
this is really an orthogonal issue.

Andrew

[Jonas Sicking]

On May 16, 2013 12:20 PM, "Andrew Sutherland" <asuth...@asutherland.org>

wrote:
>
> On 05/15/2013 07:11 AM, Mounir Lamouri wrote:
>>
>> We are starting the implementation of the API and Gaia will be involved
>> to do some experimentations so hopefully our frontend team will have a
>> solution for those UI issues.
>
>
> All of the examples for the DataStore API (
https://wiki.mozilla.org/WebAPI/DataStore) that I've seen seem pretty
specific to the contacts use-case where the upstream source of the
information has a terms of service that means we aren't allowed to freely
integrate the data with the local contacts DB. For example, Facebook.
>

> It seems like it would be better to prioritize the more-generic "Inter
App Communication" API (
https://wiki.mozilla.org/WebAPI/Inter_App_Communication) and let the
Contacts app evolve a protocol before trying to standardize it since there
is just the one planned user.
>
> This would also benefit other apps like e-mail for which the DataStore is
insufficient but the ability to expose data to another app would allow a
neat class of apps to flourish. Especially if the inter-app-communication
API allowed it to place strong requirements on other apps that might access
it. For example, if the other app could be required to be 100% sandboxed
so that it can only access its locally-stored resources defined to be part
of the app and make no network requests, concerns about information leakage
could be largely side-stepped. (Presumably more useful apps would need the
ability to trigger a web activity, which could be a means of surreptitious
information egress, but would at least require user to broadly approve the
action.)

Could you explain in more detail what the use cases you have in mind for
the email app are?

/ Jonas

[Jonas Sicking]

On Fri, May 17, 2013 at 11:58 AM, Andrew Sutherland
<asuth...@asutherland.org> wrote:
> On 05/17/2013 07:38 AM, Mounir Lamouri wrote:
>>
>> Contacts is one use case but there are other. We would like to use that
>> for messaging too: SMS and MMS such as other kind of messages could be
>> stored in a "Message" DataStore.
>
> But we already have mozMobileMessage and it exposes a richer data structure
> than the DataStore API is capable of representing (threads), and supports
> richer querying (via nsIDOMMozSmsFilter) that seems more flexible.

The Contacts and MobileMessage APIs are richer in the sense that they
support things like richer querying, like filtering, sorting and
grouping.

However the Contacts and MobileMessage APIs has the severe shortcoming
is that you are forced to live with the limitations of what querying
capabilities those APIs have. Including the performance of those
quering API.

We are *constantly* having to revise these APIs because it turns out
that the querying capabilities aren't matching what our apps need.
This is not a workable long term situation. And it's not even a
workable short-term solution for 3rd party apps since we can't revise
the APIs to support the capabilities that every 3rd party app
developer needs.

This is why the DataStore API allows applications to synchronoize data
into a application-local cache. This cache can be
stored/index/grouped/sorted in whatever format the application needs
in order to support its UI. It even allows things like merge data from
the MobilaMessage API and the Contacts API into a single location.

A very common reaction to this is "caching data in the application
means duplicating the data!". This is true, but the current path we're
on also causes a lot of duplication. Supporting all types of
filtering, sorting and grouping like we do, forces us to create a lot
of indexes. Indexes are effectively partial copies of the data. And we
have to create those indexes whether they are actually used or not,
because the API requires them.

By allowing applications to cache the data, only data that is actively
needed by installed application is copied. And we enable applications
to cache data in more formats than we could every think of and bake
into the API.

This obviously doesn't mean that DataStore API will solve all of our
problems. I suspect that something like the inter-app communication
API will still be needed.

/ Jonas

[Andrew Sutherland]

On 05/17/2013 03:06 PM, Jonas Sicking wrote:
> Could you explain in more detail what the use cases you have in mind for
> the email app are?

Short answer, it allows us to create what amounts to an extension API
for a very low cost. The e-mail back-end is already capable of
supporting multiple instances of the MailAPI interface which talks to it
over a postMessage-bridge, see:
https://github.com/mozilla-b2g/gaia-email-libs-and-more/blob/master/data/lib/mailapi/mailapi.js.
Add that, and other apps have the capability to create completely new
UIs from scratch, provide supplemental functionality, and potentially
even integrate directly into the e-mail app on a card-level basis if we
put some hooks in the e-mail app.

BIG NOTE: I am not saying this is a high priority or any priority, but
it is something that would be made much easier by the existence of the
inter-app communications API. This is blue sky talk, here.


Longer:

== Completely new UI's. (David Ascher of labs was actually asking about
something like this):
* Inbox triage/inbox zero style UI focused on helping you get things out
of your inbox.
* Enhanced search UI, possibly allowing experimental full-text search.
For example, the faceted search with timeline visualization we
implemented for Thunderbird:
http://www.visophyte.org/blog/2009/09/03/sos-your-facet-faceted-global-search-for-mozilla-thunderbird/
* Fire hose skim helper. Read through your many mozilla mailing lists
in a continuous fashion. Mark messages you think other people should
read too, posting the message-id's to a github gist. If you are busy,
only read the messages other people have starred (after scraping their
list of gists and importing the message-id's.)
* Weird visualizations of your e-mail like a radar-inspired diagram
http://www.visophyte.org/blog/?p=226
* Summaries of your folders, like the Thunderbird "Mail Summaries"
add-on that tells you your top correspondents, visualizes relative
folder size, and visualizes the times of day you send e-mail messages as
a bar graph.


== Supplemental functionality
Specifically, functionality that enhances the e-mail app by doing very
specialized things the e-mail app doesn't provide yet or will likely
never provide because they are too specialized:

* Fancy signature editor. (We'll get one of those soon, but we wouldn't
likely help you create ASCII art in your signature.)


== Extension functionality
If the e-mail app knows about the existence of apps providing extension
functionality, it could use web activities to trigger them, especially
if it is able to trigger a specific app without bringing up the prompt.
For example, the compose card could be replaced by triggering a web
activity which could be provided with the id of the message being
replied to or forwarded, to allow the extension app to create an
appropriate composition context. An extension could provide
experimental HTML composition support before we get to it, for example.

This could also be made conditional on some limited filtering support.
For example, sdwilsh's
https://addons.mozilla.org/en-US/thunderbird/addon/bugzilla-helper/ lets
you reply to bugmail from within Thunderbird.

Additional inter-app-communication channels could also be used. For
example, custom summarization of messages could be supported, like
extracting a better snippet from bugmail or even providing custom
mozNotifications when you get bugmail on a leo+ blocker. Extension apps
could badge some extra info onto the message reader, like Fabrice's
https://addons.mozilla.org/en-US/thunderbird/addon/bugmail/ which
displays the current status of a bug in the Thunderbird message reader's
header.

Andrew

[Andrew Sutherland]

On 05/17/2013 05:35 PM, Jonas Sicking wrote:
> We are *constantly* having to revise these APIs because it turns out
> that the querying capabilities aren't matching what our apps need.
> This is not a workable long term situation. And it's not even a
> workable short-term solution for 3rd party apps since we can't revise
> the APIs to support the capabilities that every 3rd party app
> developer needs.

This reply addressed all my questions/concerns. Thanks for the very
clear explanation! I have included it inline on the DataStore wiki page
at https://wiki.mozilla.org/WebAPI/DataStore#Why_not....3F and also
linked to the message on google groups in order to help clear this up
for others as well. (Feel free to integrate it into the page better, of
course! :)

Andrew

[Jonas Sicking]

In the use cases that you bring up, it seems like the 3rd party app
that uses the inter-app communication API to talk to the mail app
won't work at all if the mail app isn't installed.

This is fine in the case when it's the mail app that exposes a service
to 3rd party apps since the mail app is part of the system and thus
will always be installed.

However it doesn't work in scenarios where the service is provided by
a 3rd party app that the user may or may not have installed.

So the use cases here doesn't so much need an inter-app communication
API, but rather a system-to-app communication API. I.e. just a way to
enable us to implement APIs in gaia rather than in apps.

Another way to look at it is if there was a way to write services
which any application could declare as a dependency. That way an app
that wants to create a new email UI could import an email service. If
one already exists on the system then that is used, if not, the
service is automatically installed. That way the new email app could
depend on the service being available.

/ Jonas

[Andrew Sutherland]

On 05/18/2013 10:51 PM, Jonas Sicking wrote:
> In the use cases that you bring up, it seems like the 3rd party app
> that uses the inter-app communication API to talk to the mail app
> won't work at all if the mail app isn't installed.

Yeah, I think most of the examples would be sufficiently tightly coupled
that the presence of the Gaia e-mail app or some other e-mail app
exposing a similar-ish API would be required. (Programmers tend to have
no problem writing shims/wrappers...)

> Another way to look at it is if there was a way to write services
> which any application could declare as a dependency. That way an app
> that wants to create a new email UI could import an email service. If
> one already exists on the system then that is used, if not, the
> service is automatically installed. That way the new email app could
> depend on the service being available.

As long as Gaia apps aren't uniquely privileged to implement services,
this sounds good and enables a better user experience. But I would want
to avoid creating a situation where only the Gaia e-mail app or its
back-end is the only possible email service. Even though other apps
(with permission to use mozTCPSocket) can use any e-mail library they
want, if they are unable to expose data to each other, they would lose
out because there's a ton of overhead/busywork to syncing e-mail and no
one wants N apps duplicating that effort. Note: I think this is what
you are proposing too, I just want to be explicit about it.

Andrew

[Jonas Sicking]

I definitely agree that it can't just be gaia apps that can implement
services. I had imagined services being a separate thing from apps
entirely. And so just like anyone can write a 3rd party app, anyone
could write a service.

I think there's a lot of unanswered questions here. DataStore,
WebActivities and Inter-App communication all have in common that they
seem to work well when usign or augmenting services that built-in apps
provide. I.e. DataStore seems to work well to enable gmail or facebook
to augment the built-in contacts list. WebActivities allow anyone to
provide an alternative to the built-in camera app. Inter-App
communication could work well to enable a 3rd party app to provide new
UI for emails, or provide access to additional email servers.

However it will be very interesting to see if we're able to enable
3rd-party-to-3rd-party communication through any of those APIs. I.e.
if an ecosystem can be built that allows applications to become
greater than the sum of the parts. It's quite possible that we need to
modify these features to make that happen.

Would be very interested to hear people's thoughts.

/ Jonas

[Mounir Lamouri]

On 20/05/13 06:44, Jonas Sicking wrote:
> I definitely agree that it can't just be gaia apps that can implement
> services. I had imagined services being a separate thing from apps
> entirely. And so just like anyone can write a 3rd party app, anyone
> could write a service.
>
> I think there's a lot of unanswered questions here. DataStore,
> WebActivities and Inter-App communication all have in common that they
> seem to work well when usign or augmenting services that built-in apps
> provide. I.e. DataStore seems to work well to enable gmail or facebook
> to augment the built-in contacts list. WebActivities allow anyone to
> provide an alternative to the built-in camera app. Inter-App
> communication could work well to enable a 3rd party app to provide new
> UI for emails, or provide access to additional email servers.
>
> However it will be very interesting to see if we're able to enable
> 3rd-party-to-3rd-party communication through any of those APIs. I.e.
> if an ecosystem can be built that allows applications to become
> greater than the sum of the parts. It's quite possible that we need to
> modify these features to make that happen.
>
> Would be very interested to hear people's thoughts.

As I was seeing it, services would be implemented by allowing
applications to run in the background and by having an inter app
communication API. With those two features, any application can provide
an API that other applications will be able to use to communicate with
each other.

I do not think Web Activities will ever be the solution to that unless
we end up merging Web Activities and IAC. Regarding DataStore, it is
definitely not trying to solve the same problem in my opinion.

--
Mounir

[Francisco Jordano]

Hi everyone,

I agree that Web Activities are not the solution for everything, but IMHO, I see the DataStore api as an subset of what the IAC api can do.

For me, the ideal solution could be having the IAC api having a interface closer to the current DataStore api, related to http verbs, and allowing to use this api for both fetching content and executing operations.

Some examples that come to my mind are:

provider.get('/contacts/facebook'); // Fetch content from contacts
provider.post('/contacts', { ... object with the contact definition ...}); // Create a new contact

provider.post('/email/', { ... email definition ...}); // Sends an email with the parameters sent

provider.get('/userdictionary/w'); // Gets the words from user defined dictionary starting with W, if you change keyboard app that diccionary should remain
provider.delete('/userdictionary/awesometastic'); // To delete a specific word.

IMHO, as 3rd party developer would like the original app that offers the information to handle the data, taking care by myself of syncing data is extra work. Despite that I understand what Jonas says about users worried about performance doing their own stuff and indexing by copying data, it still fits perfectly with the idea of IAC providing the data.

As you could see on the examples above, I used the variable name 'provider' not in vain ;), both apis to me are quite familiar to android providers, something that was proposed 1 year ago but was definitely a bit too ahead.

Another point that comes to my mind is the ability to subscribe to changes that happens on the original source. Imagine the following use case, you are a music recommendation app, that wants to listen to a Datastore/IAC/Provider that notifies you whenever you add a new song to your music library:

provider.subscribe('/music/newsong', function onNewSongAdded(evt) { ... });

With that option I think we could close the circle around kind of communications between apps.

Regarding the security model for that proposal, I like the way permissions works right now in FirefoxOS, could we allow apps that expose content to define new permissions, letting the user allow or disallow the right to use it? Does that makes sense at all?

Regards,
Francisco.

[Fernando Jiménez]

Hi folks!

After talking with Francisco and Antonio we mostly agreed about the need of the DataStore API in addition to an inter-app communication API. An API that allows several apps to populate and consume a shared data store without having dependencies between each other and without the need of executing code from the owner of the data store would be really cool.

However, this API does not seem to be enough for use cases where an app wants to expose both: data and functionality. Or only processed data (from the local cache of an existing data store, for example) without worrying about the data store itself. Or only functionality (like the email examples explained by Andrew). So I would vote for an API that provides both capabilities at once :). In the end, both data sharing and functionality delegation seems to have common security issues.

I like the concept of the Inter-App Communication API, but I also really like the idea of having this API functions related to HTTP verbs as Francisco mentioned before. Why not allowing apps to expose a REST API to be consumed by other apps? It would be great to have an API that allows web apps to easily expose web services in a secure way. I am not sure if this could be an alternative to the proposed IAC API or yet another API for inter app communication, but I would love something like (trying to elaborate Francisco's examples a bit more):

// The first parameter is the origin of the app exposing the web service. The origin could also be an http(s) origin.
// The second parameter is an array of API functions to be consumed.
// **The user will be explicitily asked to allow permissions for these functions**.
let webService = navigator.getWebService("app://email.gaiamobile.org", ["send", "emails"]);

// Send a new email.
webService.post("send", { ... });

// Get email with id.
let email = webService.get("emails/id/123");
// Get emails from ferjm...@gmail.com
let emails = webService.get("emails/from/ferjm...@gmail.com");
// Get emails received since 23 May 2013.
let emails = webService.get("emails/latestEmailsSince/1369305573");

// An app could also subscribe to notifications about changes in the content shared by
// the web service or events related to its functionality.
webService.subscribe("new-email", function onNewEmail() {...});

// So the app exposing the web service could notify about these changes/events.
navigator.notifyWebServiceObservers(["new"], {...});


The manifest of the app exposing the web service could include something like:

"webservice": {

"api": {
"send": {
"allowed-actions": ["post"],
"allowed-apps": "privileged",
"permission-details": "Allow apps to send new emails"
},
"emails": {
"allowed-action": ["get"],
"allowed-apps": "installed",
"permission-details": "Allow apps to read the emails data store"
}
},

"notifications": ["new"]
}

I'm aware about the above example having its own limitations like that a consumer is required to specify the origin of the web service. However, this could be solved if apps could associate a keyword to its web service (similar to what the IAC API proposes). For the above example, the manifest could change to something like:

"webservices": [{
"email": {
"api": {...},
"notifications": [...]
}
}]

So consumers could request web services like:

navigator.getWebService("email", ["send", "emails"]);

In this case, the user will need to choose from a list of apps exposing a web service with the keywork "email" and also accept the permissions required.

The problem with this "keyword" approach is that other apps exposing an "email" web service will need to expose the same API, unless we provide a way to describe the web service API or we just trust on the developers self-interest.

For privileged apps that want to provide a keyword/interface defined on a certified/system apps, we could 'police' the apps and force the same interface. But we believe that since the third party developers' best interest would be to ape the certified app interface, since that way his app could act as a replacement app. And the same works for interfaces defined by third party apps. Ecosystem would (and should) just self-regulate.

There's another problem with the keyword approach, and the "developer's best interest" though and it's one problem we're currently having with WebActivities. On a system like this there's no control measure to restrict someone that's just malicious to develop an app that register a webservice with the same name than a system one, to get access to some data it should not get, or just to DoS the user. To avoid that, a simple solution would be to allow the caller to specify which kind of provider he wants:

navigator.getWebService("contacts", ["facebook"], "certified"); // I want a system/certified provider for this service.
navigator.getWebService("email", ["send", "emails"], "privileged"); // I want a provider that's at least privileged.
navigator.getWebService("unimportantService", ["someEndpoint"], "*"); // And for this one I don't care, any will do

Apologies for the long email and for desecrating the IAC API proposal with yet another proposal for inter app communication. I would understand if you just ignore my email ( http://www.topito.com/wp-content/uploads/2013/01/code-16.gif ) (please, don't :)).

Cheers,

/ Fernando

> _______________________________________________
> dev-webapi mailing list
> dev-w...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-webapi

[Salvador de la Puente González]

Hello people!

It is a very interesting proposal the REST approach. I proposed a
similar approach some months ago in the develooper list. You can see it
here:
https://groups.google.com/d/msg/mozilla.dev.gaia/RNEvGgUGDQE/MVEiup54tGwJ

Copy & paste (and some tweaks):

I would want to propose an extension for `app://` protocol to implement
`content providers`. Briefly, when you are accessing a `content
provider` URL, this does not return until a `HTTPResponse` object is
dispatched from the target URL or timeout. This object fakes a HTTP
response allowing the communication to be transparent. The actor
interested on content could ask for content providers covering some
"MIME" type.

It could be implemented as follows:

In the provider:
* Add 'content providers' URL to the manifest. It includes a MIME
type of the content provided, an URL as entry point and an optional
JavaScript handler.
* Use navigator.mozSetMessageHandler('apprequest', callback) to
listen for an app:// request. The request has the same form as an HTTP
request.
* Use var response = new HttpResponse(httpCode[, data]);
response.send()

In the client asking for content:
* Use navigator.getContent(mimetype[, arguments]) to ask the
browser agent for registered content providers. It receives a list of
URL suitable to answer with the content type requested.
* Use a normal XMLHttpRequest on that URL to perform an app://
request in the same way you would do with an http:// request.

This is a generalization of that described by Francisco and Fernando. We
could provide RESTful functionality on the top of this and with the use
of system messages we keep the value of not being listening all the time. ;)

Subscriptions can be implemented this way as well. Example with
subscriber and publisher:

Subscriber
1- Make an `app:// request` with verb POST to an standard entry point
app://publisher.gaiamobile.org/subscriptions and payload {channel:
'contacts', url: 'app://subscriber.gaiamobile.org/handle-contact'}

Publisher
2- It is woke up by the system receiving the `app:// request`; reads
the payload and store the channel and the url.
3- Suddenly, the publisher want to notify something
4- Make another `app:// requet` with verb POST and send it to each
subscriber url of the proper channel. In the payload it includes the
proper info.

Subscriber
5 - It is woke up by the system receiving the `app:// request`; reads
the payload and perform any operation it require.

Event, we could push for some kind of REST schema to describe RESTful
APIs (there is some initiative such as
https://github.com/wordnik/swagger-core/wiki).

What do you think about?
Hope it helps!

>> _______________________________________________
>> dev-webapi mailing list
>> dev-w...@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-webapi
> _______________________________________________
> dev-webapi mailing list
> dev-w...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-webapi
>
>

________________________________

Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo.
This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at:
http://www.tid.es/ES/PAGINAS/disclaimer.aspx

[Andrew Sutherland]

On 05/23/2013 10:55 AM, Fernando Jiménez wrote:
> I like the concept of the Inter-App Communication API, but I also really like the idea of having this API functions related to HTTP verbs as Francisco mentioned before. Why not allowing apps to expose a REST API to be consumed by other apps? It would be great to have an API that allows web apps to easily expose web services in a secure way. I am not sure if this could be an alternative to the proposed IAC API or yet another API for inter app communication, but I would love something like (trying to elaborate Francisco's examples a bit more):

The REST model can be mapped onto a MessagePort. So I think the key
question would be:

1) What benefits does making the REST model explicit provide? The key
upside I could imagine would be that debugging tools can benefit from
more explicit structure. For example, D-Bus has the Bustle UI
(http://www.willthompson.co.uk/bustle/) and dbus-monitor. However, it's
also not all that hard for a debugging tool to let you provide it with
property names that it can use to map into common idioms, like just
telling it what attribute(s) to concatenate to generate a unique request id.

2) What downsides does it have? Using your examples:

> // Get email with id.
> let email = webService.get("emails/id/123");
> // Get emails from ferjm...@gmail.com
> let emails = webService.get("emails/from/ferjm...@gmail.com");
> // Get emails received since 23 May 2013.
> let emails = webService.get("emails/latestEmailsSince/1369305573");

Note: I'm assuming these would be DOMRequests and not synchronous.

At least for the e-mail use-case, having to encode these requests into a
path or URL form would not be helpful. It just adds potential encoding
and decoding bugs on both sides.

Now, e-mail could largely use paths to replace the 'type' field we put
in the messages we send across our worker postMessage bridge, but the
model needs to be bi-directional, not a request/response model.
Specifically, for the e-mail app, when you get a piece of information,
you are largely getting a subscription to that information and any
updates. The messages you are looking at could get starred/flagged on
another e-mail client, etc. The back-end needs to be able to send this
information to the front-end, and if we end up having to implement
long-poll HTTP requests again, then that's a big step back.

> // An app could also subscribe to notifications about changes in the content shared by
> // the web service or events related to its functionality.
> webService.subscribe("new-email", function onNewEmail() {...});
>
> // So the app exposing the web service could notify about these changes/events.
> navigator.notifyWebServiceObservers(["new"], {...});

Unless this is implemented as a generic persistent message queue
implementation, I worry about this being a performance foot-gun given
current and planned hardware. Specifically, we definitely don't want to
try and wake up N apps in parallel from a broadcast. Is there any reason
apps couldn't just create a persistent subscription with the app in
question and pull the information out the next time it is run via
Inter-App Comm or DataStore?

Actually, maybe this just needs a better use-case example? I definitely
would not implement a broadcast on new emails from the e-mail app. I
know Telefonica UX had proposed a unified communications log which is a
good use-case, and a good example where good UX requires the
communications log to have heard about the information before the app
opens. But I think we would want the e-mail app to filter the
notifications to just contacts in the address book, and effectively
aggregate the information and have it provided to the communications log
app in a batch. The DataStore API could also work for this use-case,
it's just a question of when the communications log wakes up and pulls
the data.

Andrew

[Fernando Jiménez]

Thanks for your feedback Andrew!

On 23/05/2013, at 22:20, Andrew Sutherland wrote:

> On 05/23/2013 10:55 AM, Fernando Jiménez wrote:
>> I like the concept of the Inter-App Communication API, but I also really like the idea of having this API functions related to HTTP verbs as Francisco mentioned before. Why not allowing apps to expose a REST API to be consumed by other apps? It would be great to have an API that allows web apps to easily expose web services in a secure way. I am not sure if this could be an alternative to the proposed IAC API or yet another API for inter app communication, but I would love something like (trying to elaborate Francisco's examples a bit more):
>
> The REST model can be mapped onto a MessagePort.

Indeed!

> So I think the key question would be:
>
> 1) What benefits does making the REST model explicit provide? The key upside I could imagine would be that debugging tools can benefit from more explicit structure. For example, D-Bus has the Bustle UI (http://www.willthompson.co.uk/bustle/) and dbus-monitor. However, it's also not all that hard for a debugging tool to let you provide it with property names that it can use to map into common idioms, like just telling it what attribute(s) to concatenate to generate a unique request id.

IMHO there are two key benefits of making the REST model:

1) REST APIs are a very familiar and natural language for web developers, which I guess is an important point for a web API :).

2) It would allow us to build a more granular than a read/write permissions security mechanism on top of it. A generic "allow connection" security feels like insufficient in this case.

For example, for a communications log application that exposes a web service I might want to allow other apps to add new logs and also allow them to read the whole list of logs added by other apps. However, I might not be interested in allowing them to delete or modify logs, specially the logs that are not added by them. So I could do something like:

"webservice": {
"new": {
"allowed-actions": ["post"],
"allowed-apps": "installed",
"permission-details": "Allow apps to add new logs"
},
"logs": {
"allowed-actions": ["get"],
"allowed-apps": "installed",
"permission-details": "Allow apps to read the logs"
}
}

With a read/write permissions model I could only do:

"webservice": {
"new": {
"allowed-actions": ["write"],
...
},
"logs": {
"allowed-actions": ["read"],
...
}
}

But "write" would also allow apps to modify or delete logs.

So I thought that having a wider match between actions and permissions would be a good benefit for it.

> 2) What downsides does it have? Using your examples:
>
>> // Get email with id.
>> let email = webService.get("emails/id/123");
>> // Get emails from ferjm...@gmail.com
>> let emails = webService.get("emails/from/ferjm...@gmail.com");
>> // Get emails received since 23 May 2013.
>> let emails = webService.get("emails/latestEmailsSince/1369305573");
>
> Note: I'm assuming these would be DOMRequests and not synchronous.

Absolutely! That was the idea. Sorry about that. Not sure what I was thinking about...

> At least for the e-mail use-case, having to encode these requests into a path or URL form would not be helpful. It just adds potential encoding and decoding bugs on both sides.

The paths could be optional. We could also send a JSON object with "get" requests.

> Now, e-mail could largely use paths to replace the 'type' field we put in the messages we send across our worker postMessage bridge, but the model needs to be bi-directional, not a request/response model. Specifically, for the e-mail app, when you get a piece of information, you are largely getting a subscription to that information and any updates. The messages you are looking at could get starred/flagged on another e-mail client, etc.

I'm not sure that I am getting your point, but cannot this be done with "post" requests?

> The back-end needs to be able to send this information to the front-end, and if we end up having to implement long-poll HTTP requests again, then that's a big step back.

I didn't mention anything about HTTP requests. Sorry if I wasn't clear about this in my previous description. The fact that I am proposing HTTP verbs for this API doesn't mean that I am proposing to trigger any HTTP request with them.

>> // An app could also subscribe to notifications about changes in the content shared by
>> // the web service or events related to its functionality.
>> webService.subscribe("new-email", function onNewEmail() {...});
>>
>> // So the app exposing the web service could notify about these changes/events.
>> navigator.notifyWebServiceObservers(["new"], {...});
>
> Unless this is implemented as a generic persistent message queue implementation, I worry about this being a performance foot-gun given current and planned hardware. Specifically, we definitely don't want to try and wake up N apps in parallel from a broadcast. Is there any reason apps couldn't just create a persistent subscription with the app in question and pull the information out the next time it is run via Inter-App Comm or DataStore?
> Actually, maybe this just needs a better use-case example? I definitely would not implement a broadcast on new emails from the e-mail app. I know Telefonica UX had proposed a unified communications log which is a good use-case, and a good example where good UX requires the communications log to have heard about the information before the app opens. But I think we would want the e-mail app to filter the notifications to just contacts in the address book, and effectively aggregate the information and have it provided to the communications log app in a batch. The DataStore API could also work for this use-case, it's just a question of when the communications log wakes up and pulls the data.

That's certainly true. We shouldn't wake up N apps in parallel from a broadcast. I wasn't sure about the notification thing. In any case, an app could expose a "subscribe" function, build a persistent message queue and expose a function to request and release the contents of that queue on demand.

In fact that makes me think about my previous proposal and we could even add the option to let the app exposing the web service decide which messages can wake it up and which messages can be queued to be handled on a later execution. For the example of the comms log, something like:

"webservice": {
"new": {
"allowed-actions": ["post"],
"allowed-apps": "installed",
"permission-details": "Allow apps to add new logs",
"defer": true // This message won't wake up the app
},
"logs": {
"allowed-actions": ["get"],
"allowed-apps": "installed",
"permission-details": "Allow apps to read the logs"
"defer": false // Could be the default one. This message will wake up the app.
}
}

Cheers,

/ Fernando

[Andrew Sutherland]

On 05/24/2013 11:48 AM, Fernando Jiménez wrote:
>> Now, e-mail could largely use paths to replace the 'type' field we put in the messages we send across our worker postMessage bridge, but the model needs to be bi-directional, not a request/response model. Specifically, for the e-mail app, when you get a piece of information, you are largely getting a subscription to that information and any updates. The messages you are looking at could get starred/flagged on another e-mail client, etc.
> I'm not sure that I am getting your point, but cannot this be done with "post" requests?
>
>> The back-end needs to be able to send this information to the front-end, and if we end up having to implement long-poll HTTP requests again, then that's a big step back.
> I didn't mention anything about HTTP requests. Sorry if I wasn't clear about this in my previous description. The fact that I am proposing HTTP verbs for this API doesn't mean that I am proposing to trigger any HTTP request with them.
>

Let's say we have the "altUI" app and the "backend" app. The
communications flow would go like this.

1) altUI === request subscription to most recent messages in inbox ===>
backend

2) altUI <== replies with current set of messages === backend

(new email arrives)

3) altUI <== information about new message in subscribed window === backend

(some time later)

4) altUI === kills subscription ===> backend


In a traditional request/response model like HTTP, #3 might require us
to leave an open request (long-poll) from the altUI to the backend so we
can get that info about the new email. Now, since there's not the
traditional client (potentially behind a NAT) / server asymmetry, I
could also imagine the back-end just issuing a request to the altUI. Or
maybe you can respond to a request multiple times? I was wondering how
this would work under your model.

The other key thing is that there needs to be reliable connection
status. That subscription creates a resource burden on the backend, so
it needs to know when the altUI doesn't need the subscription anymore,
which includes cases of abnormal termination because of an OOM or
something like that. If the request model is too stateless, that might
be a problem unless (ugly) heartbeats get involved.

Andrew

[Salvador de la Puente González]

Hello again.

Or you can use a system message to wake up the application on demand. Of
course, as you and Fernando have already pointed we cannot afford waking
up N applications at the same time but we can add some intelligence to
the system message dispatcher as follows:

(AFAIK, system messages are already persistent so they are always
delivered but I don't know if this persistence survives a switch off, I
will suppose they do).

1 - Take measures of memory and schedule n <= N < T apps to be launched
2 - Launch them with high priority (to be unlikely killed by the OOP)
3 - Deliver system messages only to JavaScript workers (with no UI)
4 - Use some kind of acknowledged message as Julien suggest in thread
5 - After acknowledge, low the priority of the app or kill it directly
6 - Use a timeout value if acknowledge is never received.

Repeat until consuming the N applications.

Note as system messages are persistent, we don't even need to wake up
the application. The application could eventually be launched by the
user and the information could be receive then.

What do you think?

[Salvador de la Puente González]

Sorry, I missed this reference. It was Justin who suggest some kind of
commit / ack mechanism in:
https://groups.google.com/d/msg/mozilla.dev.webapi/diddFoHYLVs/-XoRpKmJqksJ

[Donovan Preston]

On 5/24/13 2:56 PM, Andrew Sutherland wrote:
> In a traditional request/response model like HTTP, #3 might require us
> to leave an open request (long-poll) from the altUI to the backend so
> we can get that info about the new email. Now, since there's not the
> traditional client (potentially behind a NAT) / server asymmetry, I
> could also imagine the back-end just issuing a request to the altUI.
> Or maybe you can respond to a request multiple times? I was wondering
> how this would work under your model.

I like using request symmetry. If one calls a service and wants the
service to call back when events happen, one can just pass a url
pointing back into the current app to the service and have the service
make requests on it later. This would be like webhooks in traditional
web services.

In addition to this, if apps became able to register web services and
respond to requests to them, then it would be trivial (aside from normal
operational and scaling difficulties) to add an "Agent server" which
gave each phone a permanent, static domain name on the web. If the phone
wasn't connected to the network, the agent server would just reply with
an error code indicating something to that effect. If the phone is
connected, it will keep the agent server up to date with a method to
reach it, and the agent server can just proxy directly to the service on
the phone.

The agent server model is nice because for normal static resources
accessed via get, the agent server can cache and deliver them even if
the phone is not currently connected to the network. (Assuming the cache
headers allow this)

D

[Mounir Lamouri]

Hi,

There was quite interesting feedback from Salvador, Fernando and
Francisco regarding the Inter Application Communication to make it more
Web-y.

Trying to come up with APIs that would work as well in the Web than in
our app ecosystem is something I would like to do more. Generally
speaking, I would like to see the "apps" as the Web and not do this
distinction any more in API we design.

In the past I thought about using XHR instead of the IAC API but there
are some high level issues that would need to be solved. However, I
think that we can give this a try...

Currently, the Web allows communication between different parties using
XHR. From a Web Client, you can do various things trough a REST API by
communication with a given Web Server. A Web App can be considered as a
Web Client and as a basic Web page, it can use XHR to communicate with a
given Web Server. But you can't use XHR to communicate with a Web App.
To summarise, we have:
Web Client <-- XHR --> Web Server
Web App <-- XHR --> Web Server
Web Client <-- ??? --> Web App
Web App <-- ??? --> Web App

If we want to allow XHRs to be handled by Web Apps, we need to solve two
things: getting a Web App to behave like a Web Server and having a way
to identify a Web App.

The former problem is pretty simple to use: we can have a 'xhr' system
message that would be sent to the Web App when it will get a XHR. The
message payload will contain the XHR payload and the app will have to
reply following a certain payload format, as usual.

The later problem is the real one: in the Web, we identify things using
URIs but this doesn't work that well with our application system. Hosted
applications have a URI identifier that is a manifest and their origin
can't be differentiated from a simple web page. And packaged
applications doesn't really have a URI (it is random).

Regarding, packaged applications, they are already not part of the web
and I wouldn't feel bad if we have them opt-in only when they have a
deterministic origin (ie. when the manifest has an origin set and it is
allowed by the runtime).

Hosted applications are more tricky. How to know if a XHR to
"http://example.com/foo/bar/" should go to "example.com" or the
application at "http://example.com/manifest.webapp"? My first thought
would be to say that the UA will redirect the XHR to the application if
it has an application installed under that origin but that might not
work for a couple of reasons.
First of all, there are some REST API that are available at
"api.example.com" (or another subdomain) like flickr [1] and as a
consequence, a flickr app (in www.flickr.com) wouldn't be able to handle
requests to api.flickr.com.
Secondly, the opposite situation might be true: some XHR might end up to
the application while it is not expected. This could be easily fixed by
simply forwarding the call to the actual server from the application but
it can be a source of bugs.

A proposal to solve this could then be to add a "xhr" entry in the
manifest that says which origin the application would like to "capture"
if a XHR is performed. On top of my head I believe that we could require
those origins to be same domain but if we go forward with that idea, our
security team should have a look into this.

Note that with the intent of moving the manifest to normal web pages
[2], we might end up with web pages handling XHR themselves :)

Finally, with a system like that in place, the latest feature to be
complete would be services discoverability. That might still have some
security issues related to it.

All of this written, we should consider the implication of something
like that for the general platform and I would be more comfortable with
a system that would be limited to a subset of apps to start with in
order to experiment before pushing something new and experimental to the
Web.

WDYT?

[1] https://www.flickr.com/services/api/request.rest.html
[2] http://mounir.lamouri.fr/2013/05/the-web-manifest.html

--
Mounir

>> I like the concept of the Inter-App Communication API, but I also
>> really like the idea of having this API functions related to HTTP
>> verbs as Francisco mentioned before. Why not allowing apps to expose a
>> REST API to be consumed by other apps? It would be great to have an
>> API that allows web apps to easily expose web services in a secure
>> way. I am not sure if this could be an alternative to the proposed IAC
>> API or yet another API for inter app communication, but I would love
>> something like (trying to elaborate Francisco's examples a bit more):
>>

>> // The first parameter is the origin of the app exposing the web
>> service. The origin could also be an http(s) origin.
>> // The second parameter is an array of API functions to be consumed.
>> // **The user will be explicitily asked to allow permissions for these
>> functions**.
>> let webService = navigator.getWebService("app://email.gaiamobile.org",
>> ["send", "emails"]);
>>
>> // Send a new email.
>> webService.post("send", { ... });
>>

>> // Get email with id.
>> let email = webService.get("emails/id/123");
>> // Get emails from ferjm...@gmail.com
>> let emails = webService.get("emails/from/ferjm...@gmail.com");
>> // Get emails received since 23 May 2013.
>> let emails = webService.get("emails/latestEmailsSince/1369305573");
>>

>> // An app could also subscribe to notifications about changes in the
>> content shared by
>> // the web service or events related to its functionality.
>> webService.subscribe("new-email", function onNewEmail() {...});
>>
>> // So the app exposing the web service could notify about these
>> changes/events.
>> navigator.notifyWebServiceObservers(["new"], {...});
>>
>>

>> The manifest of the app exposing the web service could include
>> something like:
>>
>> "webservice": {
>>
>> "api": {
>> "send": {
>> "allowed-actions": ["post"],
>> "allowed-apps": "privileged",
>> "permission-details": "Allow apps to send new
>> emails"
>> },
>> "emails": {

>> "allowed-action": ["get"],

>> "allowed-apps": "installed",
>> "permission-details": "Allow apps to read the

>>> _______________________________________________
>>> dev-webapi mailing list
>>> dev-w...@lists.mozilla.org
>>> https://lists.mozilla.org/listinfo/dev-webapi
>> _______________________________________________
>> dev-webapi mailing list
>> dev-w...@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-webapi
>>
>>
>
>

> ________________________________
>
> Este mensaje se dirige exclusivamente a su destinatario. Puede consultar
> nuestra política de envío y recepción de correo electrónico en el enlace
> situado más abajo.
> This message is intended exclusively for its addressee. We only send and
> receive email on the basis of the terms set out at:
> http://www.tid.es/ES/PAGINAS/disclaimer.aspx

[Andrew Sutherland]

I like the idea that web apps could provide offline caches for their
online counterparts. For example, mozillians.org has an API
(https://wiki.mozilla.org/Mozillians/API-Specification/List_Users/). If
the user doesn't have the "mozillians.org offline app" installed, all
requests to mozillians.org/api/ just hit the net or fail if offline. If
the user has the app, all requests go through the app which can before
caching and such. Seems cool.

From the Gaia e-mail use-case perspectives, this seems quite workable
as long as there's an efficient way to maintain a WebSocket-esque
connection. Just supporting WebSocket would be perfect, but allowing
the xhr requests to do a socket.io thing on top of long-polls would work
too. My only non-obvious request would be to explicitly support an
event that fires on the requestee side if the requester dies while the
XHR request is still in play. Like how node.js has a "close" event on
its response object
(http://nodejs.org/api/http.html#http_event_close_1). This would allow
the e-mail app to know when the referencing app has died and all of its
subscriptions should be closed out.

Andrew

[Antonio Manuel Amaya Calvo]

Hi there.

Starting by the end, I don't get what the problem is with hosted web
apps. Hosted web apps are server provided content, with a twist (the
twist being the extra information provided by the manifest). As such,
they're accessed by HTTP and they *already* can have REST semantics to
their heart content. There's nothing on the spec that says that a given
hosted web app serves static content. As a matter of fact, I know for
sure our own dogfooding store is a hosted web app that isn't static
content (the index.html that the manifest declares as the app entry
point is in fact a Django web app).

So, for any web app (packaged or not) to communicate with a hosted web
app, the process would be as simple as doing:

Web client <---- XHR ---> hosted web app (which is in fact a web server)
Web app <---- XHR ---> hosted web app (which is in fact a web server)

The only problem with this approach, for originating non hosted web
apps, is CORS and the web app origin. If the hosted web app enforces any
kind of origin control, since web apps origin is an iffy concept (except
for system and signed apps, which can specify a fixed origin and in any
case don't need it for this since they can use systemXHR).

As for using XHR to access packaged apps... the problem there is the H
in XHR stands for HTTP and packaged apps don't use HTTP at all, which is
the part of XHR that provides the REST semantics in fact. The app://
protocol doesn't currently have any kind of verb (you can't POST to an
app:// url, nor PUT, nor DELETE, nor anything, you just load it). Even
if we could define a new semantic for the app: protocol, or just add
some syntax sugar so:

xhr.open("POST", "app://one.app.origin/a/app/path");

meant the same as:

xhr.open("POST", "http://one.app.origin/a/app/path");

that would still leave us two problems:
* what do we do for (or as) CORS for app:// XHRs? At which level is is
enforced?,
* and we need some kind of on-device service discovery. While for normal
XHR an app would know the server it wants to connect, I think a more
normal use case for "inter-app XHR" would be just knowing the kind of
service it needs (I want a mail feed, a facebook contact, a RSS
feed,...) and I don't know nor necessarily care which concrete app (and
thus URL) is serving it.

Best,

Antonio

[Marcos Caceres]

Hi Antonio,

On Wednesday, 29 May 2013 at 21:25, Antonio Manuel Amaya Calvo wrote:

> As for using XHR to access packaged apps... the problem there is the H
> in XHR stands for HTTP and packaged apps don't use HTTP at all, which is
> the part of XHR that provides the REST semantics in fact. The app://
> protocol doesn't currently have any kind of verb (you can't POST to an
> app:// url, nor PUT, nor DELETE, nor anything, you just load it).

The semantic is GET. See the following spec, which defines `app:`:
http://app-uri.sysapps.org/

Unless I'm missing something, the other methods don't make any sense in a package app, as you can't PUT, DELETE, or do anything with a POST (as it's not a server you are communicating with).

> Even
> if we could define a new semantic for the app: protocol, or just add
> some syntax sugar so:
>
> xhr.open("POST", "app://one.app.origin/a/app/path");
>
> meant the same as:
>
> xhr.open("POST", "http://one.app.origin/a/app/path");

But what does it mean to POST to something that can't do anything with the data you POST? Is there a particular use case I'm not seeing?

>
> that would still leave us two problems:
> * what do we do for (or as) CORS for app:// XHRs? At which level is is
> enforced?,

Should be enforced at the same level, but only if the origin of the app is a named origin, such as:

app://myapp.foo.com

The CORS could continue to work as expected by allowing either "*" or "app://myapp.foo.com"

To get a named origin, the packaged application would need to be digitally signed - with proof by the developer that they own a particular origin (or some such).

Not sure where we got to on this wrt a solution, but one proposal was to have an "origin" or "host" member in the manifest, so something like:

{
"host": "myapp.foo.com"
}

Which is then used to construct the origin for both internal and external URI dereferencing.

> * and we need some kind of on-device service discovery. While for normal
> XHR an app would know the server it wants to connect, I think a more
> normal use case for "inter-app XHR" would be just knowing the kind of
> service it needs (I want a mail feed, a facebook contact, a RSS
> feed,...) and I don't know nor necessarily care which concrete app (and
> thus URL) is serving it.
>

This sounds a lot like web activities, no? As I'm new here, I'm not too familiar with them and if they are currently used in FxOS.

[Salvador de la Puente González]

Hello folks!

I'm going to answer both Antonio and Mounir. Sorry for the long e-mail.

First, I want to say that using XHR is not for the X(ML), nor the H(TTP) and it is only for the R(equest) and the use of REST supported by XHR. It is a matter of avoiding introducing new ideas but reusing pre-existent concepts. Using XHR with apps:// is an abstraction (some syntax sugar as Antonio says) and could be implemented over any other IPC mechanism.

I'm with Antonio in communicating with a hosted app via http://foo.com should actually communicate with the Internet in a normal way. CORS applies as usual here. An installed version of foo.com<http://foo.com>, app://foo.com, is another entry point meant to be attended by some other mechanism. The application requesting for data in http://foo.com trying to receive information from the app instead should modify itself in order to try first with the app, then with the Internet. There could be more transparent mechanisms such as replacing http:// by app:// and if not success, continue with http:// but at the end some, logic needs to be handled by the applications involved.

With this model, inter packaged-app communications are not problem at all since we don't mind if the URL looks human friendly (app://sms.gaiamobile.org) or not (app://{0123-4567-89ab-cdef}) only if they use app:// protocol. Here we are dealing with another different (but related) problem, the problem of automatic discovering.

For security reasons we could not allow an installed application to use resources from another one but with the manifest entry suggested by Fernando or Mounir we can offer an abstraction "that says which origin the application would like to "capture" if a XHR is performed" (quoting Mounir).

>From my point of view the 'xhr' manifest entry introduced by Mounir is very similar to that introduce by Fernando as 'webservice'. Just to remember:

"webservice": {
"api": {
"send": {
"allowed-actions": ["post"],
"allowed-apps": "privileged",
"permission-details": "Allow apps to send new emails"
},
"emails": {
"allowed-action": ["get"],
"allowed-apps": "installed",
"permission-details": "Allow apps to read the emails data store"
}
},
"notifications": ["new"]
}

Basically, it is saying we can perform POST on app://whatever/api/send and GET on app://whatever/api/emails. Allowing application to behave as web services will make attempts of trying other operations (aka verbs) in the same URIs will return errors saying "operation not supported" which is quite similar to 405_METHOD_NOT_ALLOWED. And trying to access other resources will result in other kind of error like "resource does not exist" which can be easily mapped to the 404_NOT_FOUND / 403_FORBIDDEN error codes or, depending on the case, to the CORS mechanism (note how the "allowed-apps" resembles to Access-Control-Allow-Origin CORS header).

Of course, you still need to wake up the application (via system message, for instance) and the handler should create some kind of response. In the case of XHR, you should be able to create XHResponse object which emulates a HTTP response to send the answer back to the requester.

The former webservice entry is equivalent to (draft for xhr entry):

"xhr" : [
{
"uri": "/api/send",
"handler": "/xhr_handler.html",
"allowed-actions": ["post"],
"allowed-origins": ["privileged"],

"details": "Allow apps to send new emails"
},

{
"uri": "/api/emails",
"handler": "/xhr_handler.html",
"allowed-actions": ["get"],
"allowed-origins": ["privileged"],

"details": "Allow apps to read the emails"
}

]


The last remaining point is automatic discovering. In my ideal XHR-based solution there will be some semantic-web-friendly solution so it could include RDF syntax which allows not only discover content in the device but in the WWW as well. The method could have an abstract signature like:

DOMFuture<[string]> discoverProviders(conceptToLookFor)


The result is a Future that, when accepted, contains a list of URIs so you can perform XHR on them. If there is no providers, then an empty list is returned. If something goes wrong, i.e. the user explicitly deny all the communications, then the future is rejected.

About the conceptToLookFor, I think the simplest approach is the keyword but it should be great if we can specify mimeTypes (suppose I want all PDF files) or RDF 'lookups' (for Individuals, aka Contacts in vcard [1], such like { ns: "http://www.w3.org/2006/vcard/ns#"<http://www.w3.org/2006/vcard/ns#>, entity: "Individual" }.

And that's all.

Thank you for reading until here ,) What's your feedback?
Best.

[1] http://www.w3.org/TR/vcard-rdf/

On 29/05/13 22:25, Antonio Manuel Amaya Calvo wrote:
Hi there.

Starting by the end, I don't get what the problem is with hosted web apps. Hosted web apps are server provided content, with a twist (the twist being the extra information provided by the manifest). As such, they're accessed by HTTP and they *already* can have REST semantics to their heart content. There's nothing on the spec that says that a given hosted web app serves static content. As a matter of fact, I know for sure our own dogfooding store is a hosted web app that isn't static content (the index.html that the manifest declares as the app entry point is in fact a Django web app).

So, for any web app (packaged or not) to communicate with a hosted web app, the process would be as simple as doing:

Web client <---- XHR ---> hosted web app (which is in fact a web server)
Web app <---- XHR ---> hosted web app (which is in fact a web server)

The only problem with this approach, for originating non hosted web apps, is CORS and the web app origin. If the hosted web app enforces any kind of origin control, since web apps origin is an iffy concept (except for system and signed apps, which can specify a fixed origin and in any case don't need it for this since they can use systemXHR).

As for using XHR to access packaged apps... the problem there is the H in XHR stands for HTTP and packaged apps don't use HTTP at all, which is the part of XHR that provides the REST semantics in fact. The app:// protocol doesn't currently have any kind of verb (you can't POST to an app:// url, nor PUT, nor DELETE, nor anything, you just load it). Even if we could define a new semantic for the app: protocol, or just add some syntax sugar so:

xhr.open("POST", "app://one.app.origin/a/app/path");

meant the same as:

xhr.open("POST", "http://one.app.origin/a/app/path"<http://one.app.origin/a/app/path>);

that would still leave us two problems:
* what do we do for (or as) CORS for app:// XHRs? At which level is is enforced?,

* and we need some kind of on-device service discovery. While for normal XHR an app would know the server it wants to connect, I think a more normal use case for "inter-app XHR" would be just knowing the kind of service it needs (I want a mail feed, a facebook contact, a RSS feed,...) and I don't know nor necessarily care which concrete app (and thus URL) is serving it.

"http://example.com/foo/bar/"<http://example.com/foo/bar/> should go to "example.com" or the
application at "http://example.com/manifest.webapp"<http://example.com/manifest.webapp>? My first thought

would be to say that the UA will redirect the XHR to the application if
it has an application installed under that origin but that might not
work for a couple of reasons.
First of all, there are some REST API that are available at
"api.example.com" (or another subdomain) like flickr [1] and as a

consequence, a flickr app (in www.flickr.com<http://www.flickr.com>) wouldn't be able to handle

[Thinker K.F. Li]

Salvador de la Puente González <s...@tid.es> writes:

>
> The last remaining point is automatic discovering. In my ideal
> XHR-based solution there will be some semantic-web-friendly solution
> so it could include RDF syntax which allows not only discover content
> in the device but in the WWW as well. The method could have an
> abstract signature like:
>
> DOMFuture<[string]> discoverProviders(conceptToLookFor)

I suggest to use existing api for service discovery. I am talking
indexedDB. It is not allowed to publish data to other applications
through indexedDB, so we should extend IndexedDB to make it work.

IIRC, I expect there have more and more requests for directory/name
services. Service discovery is a kind of directory service. I suggest
to use device wise IndexedDB as the basis of directory service.

I have a purposal of Device IndexedDB here

https://wiki.mozilla.org/User:Thinker/DeviceIndexedDB

It provides a way to create/open indexedDB in device wise, and defines
permissions according ObjectStore and (index, key) pairs.

My idea is the App Installer publishes service information to
DeviceIndexedDB. The applications request for authorization in
manifests for (index, key) paris. Then applications can do discovery
through IndexedDB.

[Thinker K.F. Li]

Is there any server side API for XHR so far?

The problem of XHR is its stateless. It means Apps need to maintain
sessions by them-self. It means cookie and time-out, ..., etc. Since
we are talking interactive communication between a pair of Apps,
WebSocket is more easy than XHR.

--
Sinker
--
天教懶漫帶疏狂