Intent to Ship: Plugin Power Saver Poster Images

[tomm...@chromium.org‏]

Sending this because piman@ thought it was a good idea.

Contact Emails:

gr...@chromium.org, tomm...@chromium.org

Spec:

A specific portion of this design doc:

https://docs.google.com/document/d/1GQ-B3gWyWYX1cXf_B8mTK2ELSVwHqT-5bAERZqPliCo/edit#heading=h.bea8i8zdwtn7

Summary:

Plugin Power Saver "pauses" plugins deemed to be peripheral (small, cross-origin). This feature is to allow web authors to specify a poster image to be used in the "paused" state.

Usage: 

  <object data="http://b.tommycli.com/flash.swf" type="application/x-shockwave-flash" width="400" height="100">

    <param name="poster" value="snapshot1.png" />

  </object>

Motivation:

Without a poster image, Chrome tries to automatically extract an "interesting" keyframe during a short preroll.

• Plugin start can be deferred if there's a pre-provided poster image. Better performance for users.
• If Chrome tries to extract a keyframe by analyzing frames, it could choose mediocre frame to use.
• Keyframe extraction process itself takes up CPU time.

Link to Intent to Implement:

I implemented this before I was aware of this process. I sorry. It's behind a content setting though, so no users will experience it unless they explicitly opt in.

Demo link:

Use Chrome Canary. See http://a.tommycli.com/small_only_poster.html

See here for a video: https://drive.google.com/open?id=0B-Use9bxumRNZE5Ub2VGYTk0c1E

Compatibility Risk:

I think it's minimal. The param tag is already supported. The 'poster' parameter continues to be passed to the plugin, it's just re-used for a different purpose. We could call it webkit-poster or something if you guys are worried, but I'd prefer not to. If we take the feature away, the browser will just ignore that parameter, and it shouldn't break anything.

Ongoing technical constraints:

None that I'm aware of.

Will this feature be on all platforms?

Windows, Mac, Linux, ChromeOS. Only the platforms with plugins.

OWP Tracking bug

None. Happy to make one if you guys want.

Link to entry on the feature dashboard

Also none, but happy to create also if you guys want.

[Chris Harrelson‏]

Is there a draft W3C spec for this attribute?

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Tommy Li‏]

No W3C spec. I assumed since it re-used the existing param tag (already used for passing arbitrary key values to plugins), there wasn't a need.

[Rachel Blum‏]

I suppose technically this doesn't require an intent to ship:

Quoth http://www.w3.org/html/wg/drafts/html/master/embedded-content.html#the-param-element:

The name attribute gives the name of the parameter.
The value attribute gives the value of the parameter.
Both attributes must be present. They may have any value.

We merely specified the behavior for Chrome in case you set name="poster" on an SWF object.

But we also thought it might be a good idea to let the Blink community know anyways, in case we misinterpreted the spec (or there are issues with this that we are missing)

On Fri, Feb 6, 2015 at 4:38 PM, Chris Harrelson <chri...@chromium.org> wrote:

[Nico Weber‏]

On Fri, Feb 6, 2015 at 4:35 PM, <tomm...@chromium.org> wrote:

Sending this because piman@ thought it was a good idea.

Contact Emails:

gr...@chromium.org, tomm...@chromium.org

Spec:

A specific portion of this design doc:

https://docs.google.com/document/d/1GQ-B3gWyWYX1cXf_B8mTK2ELSVwHqT-5bAERZqPliCo/edit#heading=h.bea8i8zdwtn7

Summary:

Plugin Power Saver "pauses" plugins deemed to be peripheral (small, cross-origin). This feature is to allow web authors to specify a poster image to be used in the "paused" state.

Usage: 

  <object data="http://b.tommycli.com/flash.swf" type="application/x-shockwave-flash" width="400" height="100">

    <param name="poster" value="snapshot1.png" />

How do you specify regular and high-dpi poster images?

[Rachel Blum‏]

Given that we wanted to stay within the constraints of <param>'s key-value approach as outlined by the spec, there are not too many options. Using srcset's syntax seems the rational choice.

But given that this placeholder image will likely end up under a translucent black shield, I'm not sure this is an entirely necessary feature.

[Antoine Labour‏]

On Fri, Feb 6, 2015 at 4:50 PM, Rachel Blum <gr...@google.com> wrote:

I suppose technically this doesn't require an intent to ship:

Quoth http://www.w3.org/html/wg/drafts/html/master/embedded-content.html#the-param-element:

The name attribute gives the name of the parameter.
The value attribute gives the value of the parameter.
Both attributes must be present. They may have any value.

We merely specified the behavior for Chrome in case you set name="poster" on an SWF object.

But we also thought it might be a good idea to let the Blink community know anyways, in case we misinterpreted the spec (or there are issues with this that we are missing)

One of the concerns I raised in the CR is that those parameters are passed to the plugin and interpreted by the plugin content in the SWF. Pages can pass arbitrary key/values to their plugin content, and we would be adding Chrome-specific semantics to a particular key ("poster"). I don't know if that particular name is used in the wild or not.

Antoine

[Rachel Blum‏]

I'd prefer to consider this a plugin-specific key. It should not be affected in any way for non-SWF content.

[Rachel Blum‏]

To clarify this further, passing values to SWF requires passing them via a specifically named key: https://helpx.adobe.com/flash/kb/pass-variables-swfs-flashvars.html

Yes, SWF could theoretically parse the DOM and extract values manually, but that's actively stepping outside standard operating parameters. 

On Fri, Feb 6, 2015 at 5:29 PM, Antoine Labour <pi...@google.com> wrote:

[Nico Weber‏]

I don't think writing new code that assumes low-resolution monitors makes sense.

[Rachel Blum‏]

Thoughts on supporting srcset syntax?

[Tommy Li‏]

I can get behind that (the srcset syntax). Like this right?

  <object data="http://b.tommycli.com/flash.swf" type="application/x-shockwave-flash" width="400" height="100">

    <param name="poster" value="snapshot1x.png 1x, snapshot2x.png 2x" />

  </object>

[Rachel Blum‏]

+1 - just want to make sure this is palatable to blink folk before we go ahead and implement that. Nico, WDYT?

[Nico Weber‏]

I don't have an opinion other than "it'd be good to have a highdpi story". srcset syntax provides that story.

[Philip Jägenstedt‏]

The linked design doc isn't public, so I've not really been able to
understand what this is about. I've requested access, but also
consider making it public or pasting the important bits here in this
thread.

Philip

[Philip Rogers‏]

I think this sounds reasonable with the srcset change. Because Safari has a similar feature, could we coordinate with them on this so uptake by content creators is good?

[Yoav Weiss‏]

On Tue, Feb 10, 2015 at 10:50 PM, Philip Rogers <p...@chromium.org> wrote:

I think this sounds reasonable with the srcset change.

A `srcset`-like syntax certainly makes sense here. We should probably include support for the 'w' descriptor as well. `sizes` is not needed unless we want to start loading these posters before layout is known. Do we?

We could add it later on if we'd see this to be a perf issue for pages with many videos (and therefore many poster images).

Also - `<video>` also has a `poster` attribute, which can currently contain only a single resource. We may want to add srcset capabilities there as well.

 

Because Safari has a similar feature, could we coordinate with them on this so uptake by content creators is good?

We totally should.

[Simon Pieters‏]

On Mon, 16 Feb 2015 19:56:59 +0400, Yoav Weiss <yo...@yoav.ws> wrote:

> On Tue, Feb 10, 2015 at 10:50 PM, Philip Rogers <p...@chromium.org> wrote:
>
>> I think this sounds reasonable with the srcset change.
>>
>
> A `srcset`-like syntax certainly makes sense here. We should probably
> include support for the 'w' descriptor as well. `sizes` is not needed
> unless we want to start loading these posters before layout is known. Do
> we?

I think we should not rush ahead of ourselves here. 'w' descriptor based
on "layout" is not specified for <img> yet. Is it not sufficient to
support only 'x' here, at least initially? Also, it's not clear to me why
poster images should wait with being downloaded until after layout.

> We could add it later on if we'd see this to be a perf issue for pages
> with
> many videos (and therefore many poster images).
>
> Also - `<video>` also has a `poster` attribute, which can currently
> contain
> only a single resource. We may want to add srcset capabilities there as
> well.

Can you raise this on the whatwg list or https://whatwg.org/newbug ?

>> Because Safari has a similar feature, could we coordinate with them on
>> this so uptake by content creators is good?
>>
>
> We totally should.

--
Simon Pieters
Opera Software

[Dimitri Glazkov‏]

I agree. I don't believe you're actually adding a new API surface here.

:DG<

[Rachel Blum‏]

On Mon, Feb 16, 2015 at 7:56 AM, Yoav Weiss <yo...@yoav.ws> wrote:

Also - `<video>` also has a `poster` attribute, which can currently contain only a single resource. We may want to add srcset capabilities there as well.

Side bar: <video> is the reason we called it poster :)

[Rachel Blum‏]

That was the interpretation on our side as well, but I'm glad we raised it with blink-dev anyways. I think srcset syntax is a valuable addition, even if it will land later than the original attribute syntax. (If I parse the spec[1] correctly, srcset allows the specification of a single image candidate without width- or density-descriptor, so that'll be backwards compatible to the current non-srcset syntax)

[1] http://www.w3.org/html/wg/drafts/html/master/embedded-content.html#attr-img-srcset 

[Simon Pieters‏]

On Tue, 17 Feb 2015 22:17:07 +0400, 'Rachel Blum' via blink-dev
<blin...@chromium.org> wrote:

> That was the interpretation on our side as well, but I'm glad we raised
> it
> with blink-dev anyways. I think srcset syntax is a valuable addition,
> even
> if it will land later than the original attribute syntax. (If I parse the
> spec[1] correctly, srcset allows the specification of a single image
> candidate without width- or density-descriptor, so that'll be backwards
> compatible to the current non-srcset syntax)
>
> [1]
> http://www.w3.org/html/wg/drafts/html/master/embedded-content.html#attr-img-srcset

Yes. However, it will not be backwards compatible with content that uses
spaces in the URL, or have commas at the start or end of the URL. Probably
this is not going to be an issue in this case.

[Dimitri Glazkov‏]

To make this more concrete: I propose that we treat this intent to ship as an FYI (like the ones from V8), rather than making this go through our process.

:DG<

On Tue, Feb 17, 2015 at 8:12 AM, Dimitri Glazkov <dgla...@chromium.org> wrote:

[Alex Russell‏]

I agree.

[Elliott Sprehn‏]

Do we have metrics for how often plugins use the key "poster" for their own usage though? That's my concern because it seems reasonable that flash video players would also accept a poster key. If they disagree on what the syntax is, or how to use it, you'll break those plugins.

Instead of trying to "sneak" the feature into the platform without a standard it seems like we should just add a poster attribute to <object> so it's clear this is for the browser not the plugin. That also makes it consistent with <video> which is nice.

[Rachel Blum‏]

Flash video players should not use <param> to transfer values - these are parameters for the plugin, not for the SWF code executed by the plugin. All parameters passed to the SWF code (i.e. the video player) should be passed via <PARAM NAME=FlashVars VALUE="var1=foo&var2=bar">

As far as I know, SWF code has no direct way query for PARAM tags. (That said, adding a metric is a good idea anyways - filed http://crbug.com/460272.)

 - Rachel

[Tommy Li‏]

For reference, below is the part of the design doc that deals with poster images:

I'm working on getting the whole thing cleaned up for release.

Explicitly Specified Preview Images

Ideally, the plugin explicitly specifies a preview image to use. This can be done by adding an optional poster param to the plugin embedding object tag. Here’s an example.

 <object data="http://example.com/flash.swf"  

         type="application/x-shockwave-flash">

   <param name="poster"

          value="snapshot1x.png 1x, snapshot2x.png 2x" />

 </object>

The param is named poster, as that’s the HTML5 attribute name used in video tags for the same purpose. The value of the poster param will be interpreted the same way as the srcset attribute of an img tag. This is to support high-dpi displays.

When there is an explicitly specified poster image, we should not load the actual plugin until the user clicks on the poster.

With any explicitly specified preview image, note that there is both potential and incentive for malicious web authors to specify preview images that are not representative of the actual content.

[Elliott Sprehn‏]

Why not just add a poster attribute and make this consistent with <video> ?

[Rachel Blum‏]

Because it is specific to SWF. We don't do this for any other plugins, so having this as an attribute on the object tag seems the wrong choice.

[di...@grorg.org‏]

On Saturday, February 7, 2015 at 11:35:25 AM UTC+11, tomm...@chromium.org wrote:

Plugin Power Saver "pauses" plugins deemed to be peripheral (small, cross-origin). This feature is to allow web authors to specify a poster image to be used in the "paused" state.

Usage: 

  <object data="http://b.tommycli.com/flash.swf" type="application/x-shockwave-flash" width="400" height="100">

    <param name="poster" value="snapshot1.png" />

  </object>

As mentioned earlier in this thread, the param tags are for consumption by the plugin. The Quicktime Plugin has been using this parameter since 1998.

Could you implement this by adding a poster attribute to object instead (like video)? We'd probably implement it as well if done in this manner.

Dean

[Dean Jackson‏]

we == WebKit in this case

Dean (now hopefully from my work email)

 

[Dimitri Glazkov‏]

Let's just do that.

:DG<

[Rachel Blum‏]

If it gets WK support, I'd be _very_ happy to do so - how many (and which) levers do we need to pull to get it added to the spec?

 - Rachel

[Dimitri Glazkov‏]

+hixie

[Simon Pieters‏]

On Tue, 24 Feb 2015 22:10:29 +0100, Dimitri Glazkov
<dgla...@chromium.org> wrote:

> +hixie
>
> On Tue, Feb 24, 2015 at 1:02 PM, Rachel Blum <gr...@google.com> wrote:
>
>> If it gets WK support, I'd be _very_ happy to do so - how many (and
>> which)
>> levers do we need to pull to get it added to the spec?

If it's still intended as a parameter for the plugin, then an attribute on
<object> should just work the same as a <param>. (Attributes-for-params
are non-conforming for authors though.)

"When the algorithm above instantiates a plugin, the user agent should
pass to the plugin used the names and values of all the attributes on the
element, in the order they were added to the element, with the attributes
added by the parser being ordered in source order, followed by a parameter
named "PARAM" whose value is null, followed by all the names and values of
parameters given by param elements that are children of the object
element, in tree order."

https://html.spec.whatwg.org/multipage/embedded-content.html#object-plugin

It seems to me the poster image would be able to appear earlier if the
browser was responsible for fetching it, rather than the plugin. OTOH,
maybe doing it in the browser would further complicate the already insane
processing model for <object>. :-)

[Rachel Blum‏]

Handy to know there's an officially sanctioned way to do this without right now - thank you for highlighting this!

But since WebKit is considering implementing the poster attribute as well, it might be worth spelling it out in the spec - just so we implement the _same_ attribute :) Especially since there is the question of syntax for HiDPI support.

 - Rachel

[PhistucK‏]

Do you intend that the <embed> version of things would work the same?

<embed poster="srcset syntax" src="flash.swf"/>

☆PhistucK

[Rachel Blum‏]

That would make sense :)