Activities in a world of URLs

[Julien Wajsberg]

(crossposting to dev-gaia, dev-b2g, dev-webapi,
reply-to: dev-webapi)

Hi,

In this crazy moment where we're allowed to reinvent everything, I'd
like that we take some time to think about activities.

From the start activities worked but never were quite satisfying,
especially because they use API instead of URLs, and use system messages.

Instead, we could use URLs and HTTP verbs.

We could use an URL like activity://<activity>/parameters. Gecko would
be responsible for transparently transforming this into eg
app://<app_name>/activities/<activity>/parameters (or a HTTP URL of course).

For example, instead of :

new MozActivity({
name: 'pick',
data: {
type: ['image/*', 'audio/*', 'video/*', 'text/vcard']
}
}).then(data => {
// do something with data
}).catch(e => {
// handle error
});

we could use:

fetch({
method: 'GET',
url: 'activity://pick/',
headers: new Headers({
Accept: ['image/*', 'audio/*', 'video/*', 'text/vcard'].join(',')
})
}).then(response => {
if (response.status === 404) {
// no handler for this activity
} else {
// do something with response.blob()
}
}).catch(e => {
// handle error
});


Similar things can happen for activities that share or open an element.
So instead of:

new MozActivity({
name: 'open',
data: {
type: mimetype,
filename: filename,
blob: blob,
allowSave: true
}
}).catch(e => {
// handle error
});

We could have:

fetch({
method: 'POST',
url: 'activity://open/',
headers: new Headers({
'Content-Type', mimetype
}),
parameters: new URLSearchParams({
filename: filename,
allowSave: 1
}), // I wish
body: blob
}).then(response => {
if (response.status === 404) {
// no handler for this activity
}
}).catch(e => {
// handle error
});

Then for the caller this is a lot similar.


Now for the callee this could be handled with a fetch event in a Service
Worker.

self.addEventListener('fetch', function(event) {
event.respondWith(new Promise((resolve, reject) => {
if (event.request.url.startsWith('/activity/pick')) {
resolve(ImagePicker.pickImage().then(blob => new Response(blob)));
}
}));
});


But this could just as well be handled by a server-side server.

Here are some issues I foresee:
* such a request could take long to fulfill and should not timeout,
because sometimes the request is handled by a user action.
* currently all activities come from a user action. I think we can keep
the same restriction in the ActivityProtocolHandler.


For Mozilla this would be great, because this would be a depart from
using a proprietary API.

What do you think ?

--
Julien

[David Rajchenbach-Teller]

I like the idea.
Couldn't we even go one step further and get rid of `activity` to
replace it with a web-based protocol handler?

https://developer.mozilla.org/en-US/docs/Web-based_protocol_handlers

On 02/06/15 16:23, Julien Wajsberg wrote:
> (crossposting to dev-gaia, dev-b2g, dev-webapi,
> reply-to: dev-webapi)
>
> Hi,
>
> In this crazy moment where we're allowed to reinvent everything, I'd
> like that we take some time to think about activities.
>
> From the start activities worked but never were quite satisfying,
> especially because they use API instead of URLs, and use system messages.
>
> Instead, we could use URLs and HTTP verbs.

--
David Rajchenbach-Teller, PhD
Performance Team, Mozilla

[Frederik Braun]

On 02.06.2015 16:23, Julien Wajsberg wrote:
…

>
> Now for the callee this could be handled with a fetch event in a Service
> Worker.
>
> self.addEventListener('fetch', function(event) {
> event.respondWith(new Promise((resolve, reject) => {
> if (event.request.url.startsWith('/activity/pick')) {
> resolve(ImagePicker.pickImage().then(blob => new Response(blob)));
> }
> }));
> });
>

This would require the Cross-Origin ServiceWorkers (already proposed in
a previous thread), because normal SWs only interrupt stuff on the
caller side. Right?

I see a CSRF / confused deputy problem with this, where suddenly
everything you want to serve becomes an Activity for someone else to
inter-operate with.

An explicit list of exposed functions sounds easier to work with. As
soon as you start exposing "internal" functions, you'll have to maintain
them forever or break other apps. That could turn out very bad.

[Julien Wajsberg]

Le 02/06/2015 16:31, Frederik Braun a écrit :
> On 02.06.2015 16:23, Julien Wajsberg wrote:
> …
>> Now for the callee this could be handled with a fetch event in a Service
>> Worker.
>>
>> self.addEventListener('fetch', function(event) {
>> event.respondWith(new Promise((resolve, reject) => {
>> if (event.request.url.startsWith('/activity/pick')) {
>> resolve(ImagePicker.pickImage().then(blob => new Response(blob)));
>> }
>> }));
>> });
>>
> This would require the Cross-Origin ServiceWorkers (already proposed in
> a previous thread), because normal SWs only interrupt stuff on the
> caller side. Right?

Yes, except the caller doesn't directly access the app's URL here; Gecko
does. So maybe we don't _really_ need Cross-Origin SW to work.

>
> I see a CSRF / confused deputy problem with this, where suddenly
> everything you want to serve becomes an Activity for someone else to
> inter-operate with.
>
> An explicit list of exposed functions sounds easier to work with. As
> soon as you start exposing "internal" functions, you'll have to maintain
> them forever or break other apps. That could turn out very bad.

This is the part I left out my proposal but was still in my head.
I think we still have to explicit the exposed activities in
manifest.webapp. For instance the System still needs to present a list
of application when several apps are available. And the target endpoints
need to be defined somewhere (unless there's a convention instead?).

So basically, instead of :

"activities": {
"new": {
"href": "/index.html#activity-new",
"filters": {
"type": "websms/sms",
"number": {
"pattern":"[\\w\\s+#*().-]{0,50}"
}
},
"disposition": "inline",
"returnValue": true
},
"share": {
"href": "/index.html#activity-share",
"filters": {
"type": ["image/*", "audio/*", "video/*", "url", "text/vcard"],
"number": {
"max": 5
}
},
"disposition": "inline",
"returnValue": true
}
}

we could have:

"activities": {
"new": {
"endpoint": "/activities/new",
"method": "POST",
"filters": {
"headers": { "Content-Type": "websms/sms" },
"parameters": { "number": {
"pattern":"[\\w\\s+#*().-]{0,50}"
} }
},
"disposition": "inline"
},
"share": {
"endpoint": "/activities/share",
"method": "POST",
"filters": {
"headers": { "Content-Type": ["image/*", "audio/*", "video/*",
"url", "text/vcard"] },
"parameters": { "number": {
"max": 5
}
},
"disposition": "inline"
}
}

I don't know if this makes sense?

--
Julien

[Fabrice Desré]

On 06/02/2015 07:28 AM, David Rajchenbach-Teller wrote:
> I like the idea.
> Couldn't we even go one step further and get rid of `activity` to
> replace it with a web-based protocol handler?
>
> https://developer.mozilla.org/en-US/docs/Web-based_protocol_handlers

We didn't go with protocol handlers in the first place because they
don't provide the functionality we need, which is letting the user
choose a provider among many and filtering on the parameters, not the
protocol.

Fabrice
--
Fabrice Desré
b2g team
Mozilla Corporation

[Fabrice Desré]

On 06/02/2015 09:15 AM, Julien Wajsberg wrote:

> Yes, except the caller doesn't directly access the app's URL here; Gecko
> does. So maybe we don't _really_ need Cross-Origin SW to work.

Yes, that's similar to how the new push api works.

>
> I don't know if this makes sense?

So, usually web activities trigger a UI to "do something". Can a service
worker wake up a page in some way? If so, we could in a first step keep
the existing |new Activity()| mechanism and only replace the system
message by a service worker event. At some point we could even implement
the activity registration in content!

[Julien Wajsberg]

Le 02/06/2015 18:42, Fabrice Desré a écrit :
> On 06/02/2015 09:15 AM, Julien Wajsberg wrote:
>
>> Yes, except the caller doesn't directly access the app's URL here; Gecko
>> does. So maybe we don't _really_ need Cross-Origin SW to work.
> Yes, that's similar to how the new push api works.
>
>> I don't know if this makes sense?
> So, usually web activities trigger a UI to "do something". Can a service
> worker wake up a page in some way?

I think openWindow is just meant for this [1] but it's not implemented
in Gecko yet.

[1] https://developer.mozilla.org/en-US/docs/Web/API/Clients/openWindow

> If so, we could in a first step keep
> the existing |new Activity()| mechanism and only replace the system
> message by a service worker event. At some point we could even implement
> the activity registration in content!

In my best dreams all system messages would be replaced by a service
worker event.
But in the case of activities, using HTTP verbs make sense if we think
of the various use cases.
Maybe this is a useless and not performant indirection though?

--
Julien

[Andrew Sutherland]

On Tue, Jun 2, 2015, at 10:23 AM, Julien Wajsberg wrote:
> We could have:
>
> fetch({
> method: 'POST',
> url: 'activity://open/',
> headers: new Headers({
> 'Content-Type', mimetype
> }),
> parameters: new URLSearchParams({
> filename: filename,
> allowSave: 1
> }), // I wish
> body: blob

I want to call the Blob out here for use-case purposes. (And I think
you recognize the potentially complex territory this gets into, hence
the "I wish". :)

While we could probably optimize the single-Blob-as-body case so that
the recipient receives the same underlying file-backed-Blob, etc., I
think it's important that whatever new mechanism we use has a
postMessage-style implementation where structured clone is used for
serialization and deserialization. This does not seem well aligned with
the pretend-REST API model since I don't think structured clone has a
standardized wire protocol and is instead an implementation detail that
has to remain confined within the browser.

This particularly matters for activity cases like "pick" where Gallery
could potentially return to the email app multiple File instances.
While this could conceptually be mapped to a multipart response with
each being split-out to a Blob again, I think it would be beneficial for
complex data-structures like Map and Set to round-trip. (Noting that
JSON does not understand them.)

Most solutions (navigator.connect, message ports/channels, etc.) meet
this need, but I want to make sure we don't discuss any dead-ends that
would end up being more hacky than web activities.

Andrew