UA Override List Policy

[Gervase Markham]

This is a proposed lightweight policy for adding sites to the UA string
override list.

I propose that we implement this policy, then switch the B2G default UA
back to the OS-agnostic one (no "Android"; bug 787054). If we hit
problems with specific sites, we file bugs, run through this process,
and get an override in place if appropriate. It's important for a number
of reasons to fix the B2G UA ASAP, and I suspect that if we resolve it
for v1 to include "Android", we'll never manage to get it out of there,
with all of the problems that will entail long term.

We want to balance the ability to react to problems users are
experiencing, with a requirement to check that we are aiming before
shooting, that we don't actually degrade user's UX (e.g. by bypassing a
check which kept them from a very broken site) and that we are making
sure the list does not grow without any organized efforts to shrink it
again.

Sites should be added to the list if/when:

1) An evangelism bug has been opened and the site has been contacted;

2) The site has proved unresponsive or unwilling to accommodate us (how
long we wait for this will depend on factors such as the popularity of
the site and the extent of breakage);

3) There is a specific proposed alternative UA for each broken product
which has minimal changes from the default;

4) Either: Deep testing (not just the front page) has shown that a UA
override for that UA in that product leads to a significant UX
improvement on the site; or we know that the fix works because it
restores a UA which that product had previously;

5) The override is only for the broken products;

6) The entry in the prefs file comes with a comment with a reference to
the bug in question.

Criterion 2 and criterion 4 are the only ones which could potentially
lead to significant delay. 4 is unavoidable; we don't want to be doing
an override without checking that it actually improves things. Sites
required by teams for functional testing or demoing of B2G would have a
very short timeout for criterion 2.

Sites should be removed from the list, for all active branches (I
propose: including stable and ESR), once the site has confirmed that
they have fixed it, or deep testing makes us believe they have.


Does this strike the right balance?

Gerv

[Robert Kaiser]

Gervase Markham schrieb:

> Does this strike the right balance?

I think it does. This looks like a good way to go forward for me.

And yes, IMGO, we can have very short timeouts on #2 for sites we
already know about, and esp. in the time before we finish up the first
release and are still able to remove a pref before actually shipping.

Robert Kaiser

[Jonas Sicking]

On Wed, Sep 26, 2012 at 5:55 AM, Gervase Markham <ge...@mozilla.org> wrote:
> This is a proposed lightweight policy for adding sites to the UA string

> override list. (Follow up to dev-platform; let me know if I've not spread
> this wide enough.)
>
> I propose that we agree this policy, then switch the B2G default UA back to

> the OS-agnostic one (no "Android"; bug 787054). If we hit problems with
> specific sites, we file bugs, run through this process, and get an override
> in place if appropriate. It's important for a number of reasons to fix the
> B2G UA ASAP, and I suspect that if we resolve it for v1 to include
> "Android", we'll never manage to get it out of there, with all of the
> problems that will entail long term.
>
> We want to balance the ability to react to problems users are experiencing,
> with a requirement to check that we are aiming before shooting, that we
> don't actually degrade user's UX (e.g. by bypassing a check which kept them
> from a very broken site) and that we are making sure the list does not grow
> without any organized efforts to shrink it again.
>
> Sites should be added to the list if/when:
>
> 1) An evangelism bug has been opened and the site has been contacted;
>
> 2) The site has proved unresponsive or unwilling to accommodate us (how long
> we wait for this will depend on factors such as the popularity of the site
> and the extent of breakage);
>
> 3) There is a specific proposed alternative UA for each broken product which
> has minimal changes from the default;
>
> 4) Either: Deep testing (not just the front page) has shown that a UA
> override for that UA in that product leads to a significant UX improvement
> on the site; or we know that the fix works because it restores a UA which

> that product was previously using;

>
> 5) The override is only for the broken products;
>
> 6) The entry in the prefs file comes with a comment with a reference to the
> bug in question.
>
> Criterion 2 and criterion 4 are the only ones which could potentially lead
> to significant delay. 4 is unavoidable; we don't want to be doing an
> override without checking that it actually improves things. Sites required
> by teams for functional testing or demoing of B2G would have a very short
> timeout for criterion 2.
>
> Sites should be removed from the list, for all active branches (I propose:
> including stable and ESR), once the site has confirmed that they have fixed
> it, or deep testing makes us believe they have.
>
>

> Does this strike the right balance?

This sounds like a good policy Gerv.

However I'm very worried that this is happening so late in the B2G
development cycle. The feature freeze for B2G is on friday this week,
and while we have a few features that will land after that, that list
is very short and driven by that we simply can't ship a phone without
those features (like 3rd party app updates).

When do you expect this feature to be implemented and land?

/ Jonas

[Gavin Sharp]

The site-specific UA override functionality has already landed on
mozilla-central and aurora (bug 782453). See also bug 787054 for its
intended use in b2g.

Gavin

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

[fbender]

As posted in the other thread, I'd rather setup an updateable list instead of baking this list right into the release thus being able to react faster on fixes to existing sites and newly found broken sites. See the IE7/8/9 compat mode list analogy.

However, your policy can apply to an extra (not-"hardcoded") list, too.

fb

[Lawrence Mandel]

> Sites should be added to the list if/when:
>
> 1) An evangelism bug has been opened and the site has been contacted;
>
> 2) The site has proved unresponsive or unwilling to accommodate us
> (how
> long we wait for this will depend on factors such as the popularity
> of
> the site and the extent of breakage);
>
> 3) There is a specific proposed alternative UA for each broken
> product
> which has minimal changes from the default;
>
> 4) Either: Deep testing (not just the front page) has shown that a UA
> override for that UA in that product leads to a significant UX
> improvement on the site; or we know that the fix works because it

> restores a UA which that product had previously;

>
> 5) The override is only for the broken products;
>
> 6) The entry in the prefs file comes with a comment with a reference
> to
> the bug in question.
>
> Criterion 2 and criterion 4 are the only ones which could potentially
> lead to significant delay. 4 is unavoidable; we don't want to be
> doing
> an override without checking that it actually improves things. Sites
> required by teams for functional testing or demoing of B2G would have
> a
> very short timeout for criterion 2.
>
> Sites should be removed from the list, for all active branches (I
> propose: including stable and ESR), once the site has confirmed that
> they have fixed it, or deep testing makes us believe they have.
>
>
> Does this strike the right balance?

I think this is a good list. Unless someone has a glaring issue I suggest that we run with this list and tweak it in the future if need be. As was raised in another response, I expect that as we get closer to the release we will continue to lower the bar for #2.

Lawrence

[Gervase Markham]

On 27/09/12 14:30, fbender wrote:
> As posted in the other thread, I'd rather setup an updateable list
> instead of baking this list right into the release thus being able to
> react faster on fixes to existing sites and newly found broken sites.
> See the IE7/8/9 compat mode list analogy.

That is mechanism. This thread is about policy.
http://en.wikipedia.org/wiki/Separation_of_mechanism_and_policy
:-)

Gerv