Re: Moving large amounts of data with share and pick intents

[Paul Kinlan]

Hi Rick, 

Thanks for the email.  Yes a lot of the format of the site has changed - we are going to start to build proper demonstration applications on this site which is why we moved it to ruby.

Will sort the contributing bit :)

We also have a public google group (web-i...@googlegroups.com), there are a lot more people there and some of the questions you are asking are important for other people to understand (and for me to update the faq with).

I have commented inline.

On Sun, Sep 11, 2011 at 12:26 PM, Rick Byers <rby...@google.com> wrote:

Hey Paul,

I was just poking around at your examples to better understand how you invision share and pick intents being used (funny actually - I was poking on github and then cloned your repo just after you moved everything around for ruby, took me a minute to realize what happened <grin>).  By the way, the "Contributing" link on your site doesn't work (there's no contributing section on the main page).

I was initially confused reading your description of the Share intent - wondering why the data format for image/* is a URI or URI list (rather than just the image data in an ArrayBuffer or binary string or something).  I see from your code that you use data URIs - so now I understand why the spec is written that way.  But why use URIs at all instead of just sending the raw image data?  I'm thinking of scenarios like sharing videos where the content is quite large.  Are data URIs really appropriate for that?  Even storing the entire data directly in the intent object worries me a bit, but I figure the implementation could always chunk the transfer and perhaps some progress indication could be added to the API.  But if the Share intent is spec'd to take only URIs (and lists of URIs) then the implementation is more constrained, right?  Or are you thinking of some other URI scheme which could still allow cross-origin communication (I'm not aware of any existing mechanism other than data URIs myself).

URI's give us the ability to pass data by value (as data URI's) or as a reference (a plain old url).  If the app recieves a url it can either try to request it straight from the app (Cross-Origin Resource Sharing can help) or it will proxy it via their own service.  Your point about a large video is exactly why we want to support plain url's, so the service can choose to load it however it wants.

With the share example the mime-type dictates what is being shared.  A text/uri-list is a collection of links, image/* is any image (either by value or by reference).

You also have to factor in that no browser other than Chrome/Webkit (and Opera) support structured clone so there is no sane way to get the data between app and service.  Also, when you say the raw data, in JS the services would potentially need to be able to understand URI's, Data URI's, Typed Arrays, FileRefs having client apps handle all of these will be a pain.

That being said, we will support structured clone, and as long as you can format the type filter to look like a mime-type and both client and service agree the same type you could easily pass an ArrayBuffer through or a plain activity stream.  So you might say, I want to send application/js.arraybuffer - and the service would expect to recieve an array buffer.

 

I'm also a little confused by the description of the Pink intent.  Shouldn't it say somewhere that the data is returned as a result of the intent, not supplied in the request (as for the Share intent)?

The assumption is that the same data type entered is the same that is returned.

 

Thanks!

   Rick

Thanks,

P

--
Paul Kinlan
Developer Advocate @ Google for Chrome and HTML5

G+: http://plus.ly/paul.kinlan
t: +447730517944
tw: @Paul_Kinlan
LinkedIn: http://uk.linkedin.com/in/paulkinlan
Blog: http://paul.kinlan.me
Skype: paul.kinlan

[Rick Byers]

Thanks Paul!  Responses in-line.

On Sun, Sep 11, 2011 at 4:01 PM, Paul Kinlan <paulk...@google.com> wrote:

Hi Rick, 

Thanks for the email.  Yes a lot of the format of the site has changed - we are going to start to build proper demonstration applications on this site which is why we moved it to ruby.

Will sort the contributing bit :)

We also have a public google group (web-i...@googlegroups.com), there are a lot more people there and some of the questions you are asking are important for other people to understand (and for me to update the faq with).

Excellent - I've joined it.

 

I have commented inline.

On Sun, Sep 11, 2011 at 12:26 PM, Rick Byers <rby...@google.com> wrote:

Hey Paul,

I was just poking around at your examples to better understand how you invision share and pick intents being used (funny actually - I was poking on github and then cloned your repo just after you moved everything around for ruby, took me a minute to realize what happened <grin>).  By the way, the "Contributing" link on your site doesn't work (there's no contributing section on the main page).

I was initially confused reading your description of the Share intent - wondering why the data format for image/* is a URI or URI list (rather than just the image data in an ArrayBuffer or binary string or something).  I see from your code that you use data URIs - so now I understand why the spec is written that way.  But why use URIs at all instead of just sending the raw image data?  I'm thinking of scenarios like sharing videos where the content is quite large.  Are data URIs really appropriate for that?  Even storing the entire data directly in the intent object worries me a bit, but I figure the implementation could always chunk the transfer and perhaps some progress indication could be added to the API.  But if the Share intent is spec'd to take only URIs (and lists of URIs) then the implementation is more constrained, right?  Or are you thinking of some other URI scheme which could still allow cross-origin communication (I'm not aware of any existing mechanism other than data URIs myself).

URI's give us the ability to pass data by value (as data URI's) or as a reference (a plain old url).  If the app recieves a url it can either try to request it straight from the app (Cross-Origin Resource Sharing can help) or it will proxy it via their own service.  Your point about a large video is exactly why we want to support plain url's, so the service can choose to load it however it wants.

Thanks for the explanation, but I'm not sure I completely see how this will work well in practice for secured content.  Eg., say I want to use 'Share' to transfer a large (not publicly shared) video from my Facebook to Google Docs private storage.  What's required to give Google Docs code permission to access my video securely in order to transfer it by reference?  I suppose the service could always make the video available under a new temporary URL which acts as the key, right?

The main problem I was concerned about though was supporting all these scenarios well while off-line (I personally believe app integration such as web intents is critical for ChromeOS - where there isn't/shouldn't be a host OS/filesystem to fall back on, and that we need to design for offline parity in everything we do in ChromeOS).  Even when on-line, I don't want to have to transfer my 1GB video over the network in order to get it into my video editing app when it's already cached in my video player.  I don't think there's any mechanism today for transferring a large amount of data while offline between apps via URLs (all the storage mechanisms are tied strictly to one origin, right?).  But probably any really efficient solution (i.e. one that doesn't require a copy) is going to require something new anyway.  Any thoughts on how this is best handled?

With the share example the mime-type dictates what is being shared.  A text/uri-list is a collection of links, image/* is any image (either by value or by reference).

You also have to factor in that no browser other than Chrome/Webkit (and Opera) support structured clone so there is no sane way to get the data between app and service.  Also, when you say the raw data, in JS the services would potentially need to be able to understand URI's, Data URI's, Typed Arrays, FileRefs having client apps handle all of these will be a pain.

Yeah, I see the challenge here.  I was hoping there would be one efficient type of transfer that would be promoted by the standard intents, but I'm probably thinking too Chrome-centric.

 

That being said, we will support structured clone, and as long as you can format the type filter to look like a mime-type and both client and service agree the same type you could easily pass an ArrayBuffer through or a plain activity stream.  So you might say, I want to send application/js.arraybuffer - and the service would expect to recieve an array buffer.

Hmm.  Above you said the mime-type dictates what is being shared (as opposed to how it's encoded in the data), doesn't this suggestion contradict that?  Eg, if an app wants to insert a video it would create a Pick intent with type=video/* so the user is presented with the services that have videos (we wouldn't want all services capable of transferring an ArrayBuffer of something, right?).  More significantly, how can an app indicate that it can support an efficient ArrayBuffer transfer but wants to also support the standard encodings?  

This actually brings up another question of mine - the issue of intent protocol versioning.  If in the future you decide that pick should support an additional encoding for a given mime-type, is there any solution without burdening the user with seeing "Pick" and "Pick v2" options?

 I'm also a little confused by the description of the Pink intent.  Shouldn't it say somewhere that the data is returned as a result of the intent, not supplied in the request (as for the Share intent)?

The assumption is that the same data type entered is the same that is returned.

Ok.  But it's an important part of an intent protocol to say whether the data is input and/or output, right?  I'm just saying that should be specified explicitly in the Pick protocol spec (but perhaps I'm just being anal - it's easy to deduce this must be the case from the brief description, but still it's supposed to be a protocol spec right?).

Rick