Use cases and some issues with web intents

[Ian Bicking]

Hi!  First I guess a short introduction: I work with Mike Hanson and Ben Adida at Mozilla, and over the past week we've been talking a lot about Web Intents.  I've only started looking closely at the project this week, so my apologies if I'm misinterpreting something.

In the process of discussing we created a kind of brainstorm of use cases; it's terse and sometimes more verb-oriented than user-intent-oriented: https://gist.github.com/1222633

Brainstorming how to satisfy some of the use cases using Web Intents (or based on my understanding of Web Intents) I had some concerns...



Issue: mimetypes can't be easily pluralized.

One can imagine two use cases for PICK on a profile (say, application/json+jcard): pick one person (someone to send a message to, for instance), and pick many people (some friends you want to add to a service, or CC on a message).  At the Javascript level it's pretty obvious, but when actually starting the intent you have to choose a mimetype, most of which are singular.

Suggestion: another standard action: http://webintents.org/pickmany

Just like pick, but returns multiples.



Issue: picking a profile and picking information about yourself are hard to distinguish.

That is, Intent("http://webintents.org/pick", "application/json+jcard") is applicable to "pick a contact" and "pick profile information" (since either way you are picking stuff, and the content type doesn't say anything about the user's relation to the content.

Suggestion: yet another action, http://webintents.org/pickself ?  Feel wonky.

Another is to more clearly figure out what the data means in the case of a pick, so you could perhaps do:

  Intent("http://webintents.org/pick", "application/json+jcard", {personal_profile: true})

This pattern is not well explored, and of course the content type applies to the response, not the intent data, unlike with other actions.  The meaning of {personal_profile: true} is only meaningful with the combination of both the action and the data type, which makes it difficult to document (http://webintents.org/pick isn't necessarily a great place to document data-type-specific details; or maybe it is?)

I am tempted to suggest that there are two broad categories of actions: query and do.  And that those two should be treated somewhat differently.  But maybe there's another approach...



Issue: Content types/mimetypes are not very applicable to the data I imagine passing around.

For instance, a URL without any extra information is sometimes sufficient, but adding extra information (e.g., title) is helpful.  But if you pass {url: "http://...", title: "..."} then the real content type is application/json (or json+something).  In many cases it feels like extra information should be added.  For example, you can pass text/html, but you probably need to also pass the base URL that the HTML came from.  But even a very small amount of extra data changes the type (unless you get sloppy and ignore that you aren't attaching an accurate type).

Suggestion: Formalize data and metadata.

HTTP for example has two channels: bodies and headers (and both request and response bodies get distinct content types).

Maybe this could work by specifying that the data (and response) contain a body, and all other keys are metadata.  For instance:

  new Intent("http://webintents.org/share", "text/uri-list", {body: "http://example.com", title: "Example Site"})



Issue: it is ambiguous (in the general case) whether the type refers to the intent data type, or the response data type.

Suggestion: Adding a sense of metadata helps; in the absence of a "body" key, type becomes irrelevant. 

Most pick cases might use only ad hoc intent data, e.g.:

  new Intent("http://webintents.org/pick", "application/json+jcard", {fields: ["name", "email"]})

Edit conveniently has symmetric types.  Save for instance would be convenient if it returned the URL to which the data was saved, but could also be ad hoc ({url: "http://storageservice.com/files/23958494"}).

[Rachel Blum]

Issue: picking a profile and picking information about yourself are hard to distinguish.

That is, Intent("http://webintents.org/pick", "application/json+jcard") is applicable to "pick a contact" and "pick profile information" (since either way you are picking stuff, and the content type doesn't say anything about the user's relation to the content.

Another is to more clearly figure out what the data means in the case of a pick, so you could perhaps do:

  Intent("http://webintents.org/pick", "application/json+jcard", {personal_profile: true})

This boils down to filtering the results of the pick (in the query case), or giving hints to the service (in the general case). The trouble there is that you now need to define which hints services should support, or alternatively, provide a way to signal at intent registration which hints are supported by a particular service. If this is something that matters in only a few cases, I'd rather see a new action type (pickself) than a generic mechanism that'll ultimately require negotiation between client and service on what is supported.

Maybe this could work by specifying that the data (and response) contain a body, and all other keys are metadata.  For instance:

That's the only way I can see this working on a larger scale without a large standardization effort on data types. Payload+Metadata would allow a consistent metadata structure to emerge over time, while a well-defined data structure will be hard to nail down with all interested parties.

Most pick cases might use only ad hoc intent data, e.g.:

  new Intent("http://webintents.org/pick", "application/json+jcard", {fields: ["name", "email"]})

This case should probably be handled entirely on the client side. You're not guaranteed the service honors your metadata (at least with intents as-is) anyways.

Rachel

[Paul Kinlan]

Hi Ian,

On Fri, Sep 16, 2011 at 6:37 PM, Ian Bicking <ianbi...@gmail.com> wrote:

Hi!  First I guess a short introduction: I work with Mike Hanson and Ben Adida at Mozilla, and over the past week we've been talking a lot about Web Intents.  I've only started looking closely at the project this week, so my apologies if I'm misinterpreting something.

In the process of discussing we created a kind of brainstorm of use cases; it's terse and sometimes more verb-oriented than user-intent-oriented: https://gist.github.com/1222633

I really like the use-cases, directions for instance would be pretty powerful.  Quite a few of these are above what we talked about at the last meeting.  I will fork the GIST and see if I can add some more, or more comments.

 

Brainstorming how to satisfy some of the use cases using Web Intents (or based on my understanding of Web Intents) I had some concerns...



Issue: mimetypes can't be easily pluralized.

One can imagine two use cases for PICK on a profile (say, application/json+jcard): pick one person (someone to send a message to, for instance), and pick many people (some friends you want to add to a service, or CC on a message).  At the Javascript level it's pretty obvious, but when actually starting the intent you have to choose a mimetype, most of which are singular.

Suggestion: another standard action: http://webintents.org/pickmany

Just like pick, but returns multiples.

The current list of intents that we have defined are singular and plural at the same time, that is a service could return a single item, many or none.  When documenting the intents the initial thoughts are the client app will know what it wants to do with the data and if it received more than one result they would choose at the application level what to do with it.

Android handles plurality with a new action http://developer.android.com/reference/android/content/Intent.html#ACTION_SEND_MULTIPLE - so in this case having a many style syntax seems to be something that people are doing today.

So we could make a standard action that says pickmany, I worry that if we combine that with pickself then would we get people trying build a pickmanyself action?  The thought moves to how say what data we want to work with.  Could we constrain it as so:  Intent("http://webintents.org/pick", "[application/json+jcard]") for multiple data items (in or out) and Intent("http://webintents.org/pick", "application/json+jcard") for just a single.  For sending a mixed array of data we could do Intent("http://webintents.org/pick", "[image/*]")

It should be simple for a service to register to handle both each and if we have a compact type filter syntax it might look like <intent action="" type="image/png, image/jpg, [image/*] > which could say expect single png, single jpg, or an array of any of image.

Would love to hear some thoughts about it.

Although there is nothing constraining any two parties agreeing pick and creating a pickmany.

 

Issue: picking a profile and picking information about yourself are hard to distinguish.

That is, Intent("http://webintents.org/pick", "application/json+jcard") is applicable to "pick a contact" and "pick profile information" (since either way you are picking stuff, and the content type doesn't say anything about the user's relation to the content.

Suggestion: yet another action, http://webintents.org/pickself ?  Feel wonky.

Another is to more clearly figure out what the data means in the case of a pick, so you could perhaps do:

  Intent("http://webintents.org/pick", "application/json+jcard", {personal_profile: true})

This pattern is not well explored, and of course the content type applies to the response, not the intent data, unlike with other actions.  The meaning of {personal_profile: true} is only meaningful with the combination of both the action and the data type, which makes it difficult to document (http://webintents.org/pick isn't necessarily a great place to document data-type-specific details; or maybe it is?)

If you could format the input data as an application/json+jcard, then you could use that data as a filter if you so chose.

I am tempted to suggest that there are two broad categories of actions: query and do.  And that those two should be treated somewhat differently.  But maybe there's another approach...

How do you see that working?  Is it the way that we name the intent?

 

Issue: Content types/mimetypes are not very applicable to the data I imagine passing around.

For instance, a URL without any extra information is sometimes sufficient, but adding extra information (e.g., title) is helpful.  But if you pass {url: "http://...", title: "..."} then the real content type is application/json (or json+something).  In many cases it feels like extra information should be added.  For example, you can pass text/html, but you probably need to also pass the base URL that the HTML came from.  But even a very small amount of extra data changes the type (unless you get sloppy and ignore that you aren't attaching an accurate type).

A lot of the scenarios that we worked with in regards to either richer data could be handled by actvitiy steams (which doesn't have its own mime-type) - the question then becomes do we strictly follow http://www.iana.org/assignments/media-types/application/index.html or the syntax?  If we follow the syntax then we have quite a bit of flexibility.

I think there are a couple of options, for the text/html example, the data could just be the base url and then the service application would handle that (i.e, make a fetch).

 

Suggestion: Formalize data and metadata.

HTTP for example has two channels: bodies and headers (and both request and response bodies get distinct content types).

Maybe this could work by specifying that the data (and response) contain a body, and all other keys are metadata.  For instance:

  new Intent("http://webintents.org/share", "text/uri-list", {body: "http://example.com", title: "Example Site"})

Android does something similar with Extras - I am told from different sources that this causes a lot of problems as Services never quite know what to expect and likewise apps don't know what application will be launched so the push everything in to one long string or take a best guess at what would be a good choice.

Ultimately we want to get to a point where client and service have a high degree of understanding about the data that will be transported without us having to standarise and formalise a data protocol.

If the app instead said the data type was something like application/json+url+title *or* activitystream/article then we cover a lot of the cases.

My personal preference is to use a lot of the work that has already been done defining common objects and use something like activitystream/article as the data type.  Which whilst not a strict official MIME type is of the same syntax.

 

Issue: it is ambiguous (in the general case) whether the type refers to the intent data type, or the response data type.

Suggestion: Adding a sense of metadata helps; in the absence of a "body" key, type becomes irrelevant. 

Most pick cases might use only ad hoc intent data, e.g.:

  new Intent("http://webintents.org/pick", "application/json+jcard", {fields: ["name", "email"]})

Edit conveniently has symmetric types.  Save for instance would be convenient if it returned the URL to which the data was saved, but could also be ad hoc ({url: "http://storageservice.com/files/23958494"}).

I have been saying that the return data is of the same data type that would be push in, so everything is semetric. Save being the example you would push an image, it might be a data URI but it is the same data that you would get if you visited the return URL.

One area where this might not work is how do you handle a convert action, I want to convert from image/jpg to text/plain?

P

--
Paul Kinlan
Developer Advocate @ Google for Chrome and HTML5

G+: http://plus.ly/paul.kinlan
t: +447730517944
tw: @Paul_Kinlan
LinkedIn: http://uk.linkedin.com/in/paulkinlan
Blog: http://paul.kinlan.me
Skype: paul.kinlan

[Greg Billock]

On Fri, Sep 16, 2011 at 11:54 AM, Rachel Blum <gr...@google.com> wrote:

Issue: picking a profile and picking information about yourself are hard to distinguish.

That is, Intent("http://webintents.org/pick", "application/json+jcard") is applicable to "pick a contact" and "pick profile information" (since either way you are picking stuff, and the content type doesn't say anything about the user's relation to the content.

Another is to more clearly figure out what the data means in the case of a pick, so you could perhaps do:

  Intent("http://webintents.org/pick", "application/json+jcard", {personal_profile: true})

This boils down to filtering the results of the pick (in the query case), or giving hints to the service (in the general case). The trouble there is that you now need to define which hints services should support, or alternatively, provide a way to signal at intent registration which hints are supported by a particular service. If this is something that matters in only a few cases, I'd rather see a new action type (pickself) than a generic mechanism that'll ultimately require negotiation between client and service on what is supported.

My preference is to ask the client to make this clear based on calling context. For example, if I'm picking a profile picture, we don't want a particular action for that. The case of identity is trickier, and while it seems to me that web intents may be valuable for exploring that case, and perhaps will figure in a good solution, we should leave that as a try-not-to-rule out use case, at least for now.

So in short, if I'm expected to pick my own identity hcard, then the client should be responsible for making that clear. (And perhaps I'm signing up on behalf of someone else, anyway. Should that be impossible? Perhaps in some cases it should be and in some it shouldn't be; I think that's why the identity question is so tricky.)

 

Maybe this could work by specifying that the data (and response) contain a body, and all other keys are metadata.  For instance:

That's the only way I can see this working on a larger scale without a large standardization effort on data types. Payload+Metadata would allow a consistent metadata structure to emerge over time, while a well-defined data structure will be hard to nail down with all interested parties.

Most pick cases might use only ad hoc intent data, e.g.:

  new Intent("http://webintents.org/pick", "application/json+jcard", {fields: ["name", "email"]})

This case should probably be handled entirely on the client side. You're not guaranteed the service honors your metadata (at least with intents as-is) anyways.

Rachel

The question of plural mime-types, or url+title+description, is a really good one. I don't think there's a clear sense in the syntax that the |type| identifier must be a MIME type. Paul, particularly, has argued that in order to do really good extensible matching, it should be confined syntactically to opaque string matching. Being able to support wildcards, though, is just sooo appealing for some things! :-)

In general, I think all we need syntactically is to define that the output type is a function of the input type and the action string. A simple version of that is that output and input types are equal, as Paul says. It may not be sufficient, however, for everything. Even something as directly in-scope as a SHARE intent, for example, is more useful with a different return type from the invocation type. (i.e. "share-image/png". What do I get back? A png with a success code written on it? :-)) So perhaps another simple version is "output type dictated by the action."