GWT emulation of HTML5/CSS3 features

[dflorey]

Hi,
I've been wondering how GWT should deal with upcoming new features in
HTML5/CSS3.
There are several areas where functionality that has been implemented
in GWT is now also available in the upcoming rendering engines.

GWT is creating highly optimized JavaScript and the JavaScript-engines
are getting better and better... but: My guess is that for example
animations will be smoother when using CSS3 animations instead of
JavaScript based animations.
Same about rounded corners/shadows and stuff alike. In GWT you'll
typically use DecoratedPanel to implement rounded corners with
shadows. But Firefox3.5 and the latest Safari and Chrome releases also
support css-based rounded borders and shadows.

So my proposal would be to use deferred binding to "emulate" these
features on browsers that do not support the latest features (IE8...)
and to use a lightweight css based impl on WebKit/Firefox 3.5.

In my example of DecoratedPanel the 9x9 approach should be kept for IE
and a null impl with css based rounded corners should be available for
Firefox (css have to match the given theme).
Animations that come with the standard widgets should also be able to
fallback to css based animations when available.

I've been also reading some posts about the new datagrid html
extension and thought it might be clever to have a look at the spec
when moving the tables from incubator to trunk to see how far the
concepts match. Would be very cool to have a native table
implementation on WebKit browsers while other fallback to gwt impls.

What do you think?

[Fred Sauer]

+1

--

Fred Sauer

Developer Advocate

Google Inc.

1600 Amphitheatre Parkway

Mountain View, CA 94043
fre...@google.com

[tfreitas]

+1

[nicolas de loof]

Transparent support for CSS3/HTML5 on all browsers including IE would be a killer feature !

+1

2009/7/1 tfreitas <tfre...@gmail.com>

[Joel Webber]

I'm very much on board with the idea of moving forward on new browser features by emulating them on old browsers wherever possible. This is an important benefit of using tools (code generation, deferred binding, monolithic compilation) to get leverage on this problem. I think the best approach is to consider each such feature separately. They're going to require different approaches (e.g., some can be dealt with entirely in CssResource, but others will likely require the use of specific classes), and some will turn out to be impossible, or practically so (e.g., support for the <video> tag without a transcoding server to deal with Ogg, H.264, Flash, etc). So for the specific cases described below...

CSS Animations:

  I think we need to actually measure the performance difference before deciding whether it's even worth the trouble. I suspect that there may not be much of one (though of course I could be wrong), given that it has to do the same work, possibly including layout, regardless of how it's specified. And CSS animations aren't as general as programmatic ones. That said, there might be some utility in having CssResource parse them automatically and write the necessary code for you on older browsers (though I haven't assessed the feasibility of doing so).

Rounded Corners / DecoratorPanel:

  Here I think it's going to be a practical necessity to require that a specific class (such as DecoratorPanel) be used. The reason is that, to my knowledge, there is no way to get general 9-box rounded corners without a specific DOM structure. But DecoratorPanel could be modified to take advantage of CSS3 9-boxes on newer browsers.

DataGrid:

  I honestly haven't read enough about this proposal to have any idea whether it makes sense or not. But if it is implemented and offers a substantive advantage, then we should certainly have a look.

Some others that might be interesting:

Canvas:

  This is a nasty case, because Canvas cannot be implemented sanely or efficiently on top of VML, which is the only game in IE town. Existing canvas-on-VML implementations notwithstanding -- they have wildly different performance semantics, which is pretty unacceptable in my opinion.

SVG:

  Things are a bit brighter here. There are some things (foreign objects and certain gradient patterns come to mind) that SVG can do, which VML cannot. But a sane navigation of the common features could lead to a quite usable and efficient vector graphics library. There's the existing GWTCanvas that Jaime wrote a while back as a starting point (which uses Canvas rather than SVG), but it appears to me that SVG performance has gotten a lot better since that was written, so it's probably worth reconsidering that approach.

HTML5/Gears Database:

Geolocation:

  These shouldn't be too difficult, as applications can be easily made sensitive to their presence or absence. The database/client-side storage APIs may need some cross-browser love, as there are a few different approaches and subtle differences across browsers, but I believe that is manageable.

Cross-Document Messaging:

  I'm pretty sure this can be emulated with window.name hackery.

App Cache:

  This is something we should support at the Linker level. And like the database APIs, an application can be made sensitive to its availability without too much difficulty.

CSS Transforms:

  I think we're pretty much screwed on this front. We could *try* to do translation, but I seriously doubt it's worth the trouble (and would probably cause layout issues, as the semantics are subtly different than left: and top:). But rotation and scale (not to mention arbitrary affine transforms) seem impossible to emulate.

<audio>, <video>:

I'm not terribly bullish on these. <audio> is at least theoretically supportable using Flash, and I could see something like the SoundManager2 js library taking advantage of it. But <video> is rife with codec licensing problems that seem unlikely to get resolved any time soon (If anyone wants to debate the ins and outs of codec licensing, let's *please* do so on another thread, because I can tell you from experience that the thread won't converge).

And of course, there are probably others I'm not thinking of, so feel free to chime in with ideas.

joel.

[Bart Guijt]

Another aspect to consider is how to connect an HTML5 feature to the
compiler permutations. Right now we have 5 browser permutations, and
only the latest versions of these browsers offer support for (some)
HTML5 features.

Potentially the amount of permutations accumulate significantly if
fine-grained HTML5 support is desired. What would be the best approach
here?

I am trying to setup some GWT API's for these HTML5 features at my gwt-
mobile-webkit googlecode project, and I'm still not sure what the best
approach is to the problem. You could argue that the amount of
permutations is totally unimportant since you can subset just a few of
them for testing purposes.

What do you guys think?

Bart Guijt

[Thomas Broyer]

On 2 juil, 17:03, Joel Webber <j...@google.com> wrote:
> Some others that might be interesting:
>
> Canvas:
>   This is a nasty case, because Canvas cannot be implemented sanely or
> efficiently on top of VML, which is the only game in IE town. Existing
> canvas-on-VML implementations notwithstanding -- they have wildly different
> performance semantics, which is pretty unacceptable in my opinion.

There's also the Gears Canvas (you could get a data: URL out of it and
give it in an Image for IE8+) and of course Flash (think
Chronoscope ;-) ) and/or Silverlight.

> SVG:
>   Things are a bit brighter here. There are some things (foreign objects and
> certain gradient patterns come to mind) that SVG can do, which VML cannot.
> But a sane navigation of the common features could lead to a quite usable
> and efficient vector graphics library. There's the existing GWTCanvas that
> Jaime wrote a while back as a starting point (which uses Canvas rather than
> SVG), but it appears to me that SVG performance has gotten a lot better
> since that was written, so it's probably worth reconsidering that approach.

And for IE compatibility, Silverlight again (see
http://intertwingly.net/blog/2008/01/26/SVG-Shiv and similar articles
on Sam Ruby's blog)

> HTML5/Gears Database:
> Geolocation:
>   These shouldn't be too difficult, as applications can be easily made
> sensitive to their presence or absence. The database/client-side storage
> APIs may need some cross-browser love, as there are a few different
> approaches and subtle differences across browsers, but I believe that is
> manageable.

It definitely is: http://google-code-updates.blogspot.com/2009/05/gmail-for-mobile-html5-series-common.html

> Cross-Document Messaging:
>   I'm pretty sure this can be emulated with window.name hackery.

Cannot we rather consider this an "optional feature", just like
database, geolocation and app cache?

> App Cache:
>   This is something we should support at the Linker level. And like the
> database APIs, an application can be made sensitive to its availability
> without too much difficulty.
>
> CSS Transforms:
>   I think we're pretty much screwed on this front. We could *try* to do
> translation, but I seriously doubt it's worth the trouble (and would
> probably cause layout issues, as the semantics are subtly different than
> left: and top:). But rotation and scale (not to mention arbitrary affine
> transforms) seem impossible to emulate.
>
> <audio>, <video>:
> I'm not terribly bullish on these. <audio> is at least theoretically
> supportable using Flash, and I could see something like the SoundManager2 js
> library taking advantage of it. But <video> is rife with codec licensing
> problems that seem unlikely to get resolved any time soon (If anyone wants
> to debate the ins and outs of codec licensing, let's *please* do so on
> another thread, because I can tell you from experience that the thread won't
> converge).
>
> And of course, there are probably others I'm not thinking of, so feel free
> to chime in with ideas.

Drag'n'drop! (using Gears or Yahoo! BrowserPlus as an alternative to
browser-native support –if detectable–)

And there are a few other things that have been split from HTML5 into
their own spec:
- Web Workers ("optional feature", same as database, geolocation,
etc. see above)
- WebSockets (using long-polling –eventually in a Worker– for now as
no-one implements it)
- File API (use Gears or BrowserPlus as a fallback when available)
- XMLHttpRequest 2 (progress events and ability to send binary data
from the File API; again, Gears could be used to emulate it,
BrowserPlus to a lesser extent)
- Selectors API (already supported by GQuery)

See http://www.w3.org/2008/webapps/

[Ray Cromwell]

On Thu, Jul 2, 2009 at 8:03 AM, Joel Webber <j...@google.com> wrote:

CSS Animations:

  I think we need to actually measure the performance difference before deciding whether it's even worth the trouble. I suspect that there may not be much of one (though of course I could be wrong), given that it has to do the same work, possibly including layout, regardless of how it's specified. And CSS animations aren't as general as programmatic ones. That said, there might be some utility in having CssResource parse them automatically and write the necessary code for you on older browsers (though I haven't assessed the feasibility of doing so).

It might be useful however to create an animation library that consists of declarative types, as well as custom animation types. If you compose declarative types, you potentially get CSS Animations or SMIL or HTML+TIME (IE), if you include a custom type (uses Java code) in the composition, then you get a pure programmatic fallback.

Think of it as a type-safe enumeration pattern, where some types have deferred bindings (CSS Anim, SMIL, or pure JS) and others are just pure-JS.

 

Canvas:

  This is a nasty case, because Canvas cannot be implemented sanely or efficiently on top of VML, which is the only game in IE town. Existing canvas-on-VML implementations notwithstanding -- they have wildly different performance semantics, which is pretty unacceptable in my opinion.

It's possible to use Flash, but in general, performance can't come close to matching HTML5 canvas, not just because IE Javascript is so slow, but also because marshalling/unmarshalling between JS and Flash is slow. A bright spot is Silverlight, since the JS->Silverlight call mechanism is pretty efficient. On the downside, you can't depend on Silverlight.

Canvas is also split, since some Canvas impls support text rendering and others do not.

 

SVG:

  Things are a bit brighter here. There are some things (foreign objects and certain gradient patterns come to mind) that SVG can do, which VML cannot. But a sane navigation of the common features could lead to a quite usable and efficient vector graphics library. There's the existing GWTCanvas that Jaime wrote a while back as a starting point (which uses Canvas rather than SVG), but it appears to me that SVG performance has gotten a lot better since that was written, so it's probably worth reconsidering that approach.

Brad Neuberg is actually working on a fairly complete SVG shim for IE using Flash, it appears performance and compatibility are good.

 

<audio>, <video>:

I'm not terribly bullish on these. <audio> is at least theoretically supportable using Flash, and I could see something like the SoundManager2 js library taking advantage of it. But <video> is rife with codec licensing problems that seem unlikely to get resolved any time soon (If anyone wants to debate the ins and outs of codec licensing, let's *please* do so on another thread, because I can tell you from experience that the thread won't converge).

There's also the case that doing anything more interesting with <video> other than playback means accessing frame pixel data, and shipping them from Flash back into the browser would be *VERY* inefficient and slow.

-Ray

[Arthur Kalmenson]

This would definitely be a killer feature.

A common API for something like Web Workers and App Cache (maybe
wrapper for http://code.google.com/p/webstorageportabilitylayer/) that
can seamlessly switch talk to Gears or native HTML 5 implementation
would be very nice. I think it's a lot easier to convince a company to
install Gears then installing and using a completely new browser, so
at least for enterprise settings some common API would be very useful.

Regards,
--
Arthur Kalmenson

[dflorey]

I think it is helpful if we distinguish between optional and mandatory
features.
If GWT would come with a set of widgets or library components that
behave slightly different on different browsers this might not be bad
in general. I'm aware that the general approach of GWT is to write
software once and run it almost everywhere with identical results.
But I can imagine several areas where it may be appropriate to have
e.g. widgets that look even better when the rendering engine supports
3d animations etc. If the developer can use the widget as is and will
get a "default" behavior on all browsers and an extended behavior on
top on some of them, this would be beneficial IMO.
Think of a TabPanel widget that takes advantage of the new funky 3D
animation and transition feature on WebKit and provides a working
default implementation for browsers not supporting CSS3 looking a
little bit more boring.
When it comes to libraries like Database storage it would be helpful
to have a dummy implementation that will do nothing in case when no
persistence is available and to provide some "isAvailable()" method
for each additional feature.

On Jul 22, 5:00 pm, Arthur Kalmenson <arthur.k...@gmail.com> wrote:
> This would definitely be a killer feature.
>
> A common API for something like Web Workers and App Cache (maybe

> wrapper forhttp://code.google.com/p/webstorageportabilitylayer/) that