Preliminary results of aliasing Webkit CSS properties in Gecko analysis

[Lawrence Mandel]

The Layout, QA, and Market Insight teams recently analyzed the impact of aliasing Webkit CSS properties in Gecko with the goal of answering the question,

Does aliasing a subset of Webkit CSS properties in Gecko improve mobile Web compatibility?

The results thus far indicate that there is a very small benefit of adding Webkit CSS aliases to Gecko. However, our research is not yet complete, so we will refrain from making any definitive decisions until all our research has been carried out.

The consensus from dbaron, jet, jsmith, tchung, aaronmt, jjensen, and me is that the value of aliasing Webkit CSS properties in Gecko alone appears to be pretty low and the benefit does not warrant its inclusion in the platform at this time. The following is an overview of our analysis which led us to reach this initial conclusion. There are four hurdles that we have to jump in order to receive and render Webkit content. These hurdles are listed in increasing technical cost:

1. user agent sniffing
2. Webkit CSS prefixes
3. DOM/WebAPI aliases
4. Implementation of missing CSS/DOM/WebAPI features in Gecko

The analysis we conducted focuses on #1 and #2.

Test Fennec build:
To conduct the analysis, we wanted to compare Fennec builds with and without Webkit CSS property aliases. David Baron created a Fennec build that included aliases for the following properties, most of which we have recently unprefixed (notably text-size-adjust, appearance, and user-select have not yet been unprefixed):

-webkit-animation, -webkit-animation-delay, -webkit-animation-direction, -webkit-animation-duration, -webkit-animation-fill-mode, -webkit-animation-iteration-count, -webkit-animation-name, -webkit-animation-play-state, -webkit-animation-timing-function, -webkit-appearance, -webkit-border-bottom-left-radius, -webkit-border-bottom-right-radius, -webkit-border-radius, -webkit-border-top-left-radius, -webkit-border-top-right-radius, -webkit-background-clip, -webkit-background-origin, -webkit-background-size, -webkit-border-image, -webkit-box-shadow, -webkit-text-size-adjust, -webkit-transform, -webkit-transform-origin, -webkit-transform-style, -webkit-transition, -webkit-transition-delay, -webkit-transition-duration, -webkit-transition-property, -webkit-transition-timing-function, -webkit-user-select. The build also includes the following @-rule alias: @-webkit-keyframes.

Test methodology:
Jason Smith and Aaron Train (QA) tested two lists of sites (more on the lists below) using the two Fennec builds and Safari on iPhone. In order to alleviate some of the issues due to UA sniffing, both Fennec builds were tested using their stock UA and using a spoofed iPhone UA (via the Phony add-on). The focus of this testing was to identify specific improvements to the user experience of a site, such as improved layout, appearance and function.

Site data:
John Jensen has been analyzing mobile Web compatibility issues, specifically related to UA sniffing and Webkit CSS property usage, using a tool that he wrote to scrape CSS from the top 18,000 Alexa sites. He has built up a database that provides the ability to identify sites that make use of Webkit CSS properties without using the equivalent moz prefixed or unprefixed properties. The two test runs that I will describe below were both conducted using lists created from John's site data.

The test runs:

Using the data described above, John created a list of sites that are known to use the Webkit CSS properties that David aliased in his build without using the equivalent moz or unprefixed properties. Aaron and Jason tested the top 20 sites on this list. In these tests, the Webkit CSS aliases resulted in 2 sites with improvements (Disney and Comcast), 17 sites with no noticeable difference, and one site (howtogeek.com) that actually had a degraded layout.

The results of the first run were surprising given that we had targeted the specific Webkit CSS properties that David had aliased in his build. However, the two sites that had a noticeable improvement both make heavy use of Webkit CSS animations and transitions. For our second test run John created a new list that specifically targeted sites that use Webkit CSS animations and transitions without using the equivalent moz prefixed or unprefixed properties. The expectation was that this set of sites would see a more significant improvement. For this test run Jason and Aaron tested 18 sites. In these tests, the Webkit CSS aliases resulted in 3 sites with improvements (allhiphop, Comcast, and Muslima), 1 site with a slight improvement (not enough to make the site usable), and 14 sites with no noticeable difference.

The results of Jason and Aaron's tests can be seen at

https://docs.google.com/spreadsheet/ccc?key=0AiOpueKMbyhudDVUZW5NVWt1aUJvZWFZaDlMUHJVdlE#gid=2

Summary:
While 38 sites can hardly be considered a representative sample of the Web, we produced two test runs that specifically targeted sites that we expected to improve with the use of Webkit CSS property aliasing. However, in our tests only a small percentage of the sites showed improvement with aliasing. When this small percentage improvement is put in the context of the entire mobile Web, which includes other issues such as UA sniffing and browser specific functionality, the impact of aliasing Webkit CSS properties is even smaller.

Next steps:
These results show that there is little benefit to aliasing Webkit CSS properties in Gecko alone (hurdle #2). It's also clear that jumping both #1 and #2 together is still not effective enough to effectively render the Webkit web. It is possible that aliasing Webkit CSS properties along with one or more additional tactics, such as aliasing DOM properties like event.srcElement and WebKitPoint, may provide the benefit to mobile Web compatibility that we are after. As such, we are now focusing our research on hurdles #3 and #4. Note that we may have to jump all four hurdles to accomplish the goal of receiving/rendering Webkit content from unmodified mobile sites.

QA will next take a deep dive on 5 of the sites that they previously tested to identify the complete list of issues with each site. The results of this deep dive should tell us every change that is required in order to fix the sites so that they function in Gecko. We will also try to identify the use of any Web framework and associate the issues that lie in the framework and the issues that are with site specific code.

Lawrence

[Lawrence Mandel]

Cross posting to dev-planning to ensure that the planning audience sees these preliminary results.

Please post and responses to dev-platform.