Updating the blocklist policy
Justin Scott
The current blocklist policy (https://wiki.mozilla.org/Blocklisting) has
been in place since the feature was introduced in Firefox 2, and since
then we've blocked quite a few add-ons and have a much better idea of
the circumstances under which we want to block add-ons and plugins.
I'd like to revamp this policy to provide more guidance on when we
should block and establish more specific criteria for blocking so that
it's easier to make decisions on whether to block something or not.
I've been working with Kev Needham, Christian Legnitto, Mike Morgan, and
others that regularly deal with blocklist issues on a policy draft and
am ready to propose it to replace the current blocklist policy.
Additionally, Joe Drew has written a draft policy for the blocking of
graphics drivers, a new feature in Firefox 4.
Please take a look at the drafts below and share any feedback.
https://wiki.mozilla.org/User:Fligtar/Blocklist_Draft
https://wiki.mozilla.org/User:JoeDrew/GFXBlocklistDraft
Thanks,
Justin
L. David Baron
> Please take a look at the drafts below and share any feedback.
>
> https://wiki.mozilla.org/User:Fligtar/Blocklist_Draft
This largely looks good to me -- clarifies a bunch of issues around
maybe-malware that I've been unsure what our policy was.
A few comments on this section, though:
# Acceptable reasons for blocking software include:
#
# * Critical security vulnerabilities
# * High crash volume (10,000 or more in a week)
# * Malicious in nature
# * Severe performance impact (adds more than 75% to start-up time)
# * Severe bugs that unintentionally affect core Firefox features
The "10,000 or more in a week" crash threshhold seems like it
doesn't apply well to betas. Maybe this could be phrased in terms
of crashes / ADU / day or crashes / ADU / week? (I'm not sure what
the right number is -- since I'm having trouble figuring out what
the crashes / ADU number we report actually means.)
The "(adds more than 75% to start-up time)" should presumably have
an "e.g., " in front of it, since there are certainly other things
I'd imagine blocking an addon for -- such as similar effects on page
load or rendering times.
> https://wiki.mozilla.org/User:JoeDrew/GFXBlocklistDraft
Is the distinction between downloaded blocklist and compiled-in
blocklist really as severe as this document suggests? Don't we ship
a "current" version of the downloaded blocklist with each release so
that we have something to apply on first startup?
-David
--
L. David Baron http://dbaron.org/
Mozilla Corporation http://www.mozilla.com/
johnjbarton
The proposal lists items that are clearly user-protection issues:
Critical security vulnerabilities
High crash volume (10,000 or more in a week)
Malicious in nature
Severe bugs that unintentionally affect core Firefox features
and one issue that is clearly not:
Severe performance impact (adds more than 75% to start-up time)
As a matter of policy, meaning a way of making decisions based on
principles rather than whim, this item has no place on the list.
Performance is a choice, it is like tabs-on-top, it is not like
security, crashes, harm, or unintended side effects.
To take a concrete example, consider "adds more than 75% to start-up
time". Firebug already exceeds this bar and yet we have not a single
bug report or user complaint about startup time. The reason is simple:
this issue does not matter to the vast majority of our users. When
Firebug devs make decisions on where to make improvements we have to
make choices based on our user's needs. Multiprocess support, mobile
support, memory analysis tools, JavaScript hot-code replacement,
editor integration, jsd2, extension support, CSS layout debugging,
HTML5 support, Console re-implementation, Web Worker support, and so
on are more important than start up time. Sure if fixing startup time
was easy we'd do it, but its not. And eventually, when we get to
multiprocess support, we'll have stopped using overlays and the
startup time problem will be solved.
To anticipate a counter-argument along the lines of "we're protecting
users", it's not true. Users may choose to pay 500ms start up time, or
5s or 50s for that matter, to use a extension that provides some
feature even if the Firefox team can't imagine that choice. It's
really no different than tabs on top or title bars or whatever. CPU
time is no different from screen real estate or disk space or color
palette.
Of course users may 'blame' Firefox for the extra 500ms, meaning they
did not make an informed choice to pay the 500ms. That issue can be
addressed by informing users. For example that is how Microsoft IE
handles start-up time. (The information should be accurate unlike the
current AMO effort, but that is a different topic).
jjb
johnjbarton
I object to this item:
Severe performance impact (adds more than 75% to start-up time)
Startup time is now sub-second, so 75% would be easily crossed by an
addon and yet be entirely reasonable for users.
More generally, some addons have performance impact and there is no
practical alternative or technical solutions. Examples from Firebug
include memory overhead ("1.7.0 consumed 2.5GB over night" to quote a
user), page rendering overhead (DOM mutation listeners), script overhead
and so on.
To me there should be no bar on non-malicious performance, it's between
the addon dev and the user.
jjb
Boris Zbarsky
> Startup time is now sub-second, so 75% would be easily crossed by an
> addon and yet be entirely reasonable for users.
Practice shows that users don't consider anything over about 200ms
reasonable. They may even _tell_ you that they consider it reasonable,
but then will complain about things being slow.
So I think we're pretty serious about the 75% startup time overhead
metric. I happen to think it's pretty generous, myself.
-Boris
Boris Zbarsky
> To take a concrete example, consider "adds more than 75% to start-up
> time". Firebug already exceeds this bar and yet we have not a single
> bug report or user complaint about startup time.
But _we_ have had complaints about that. And when we suggest that the
user turn off Firebug they try it, get very happy and say "Oh, is _that_
what was making things slow?"
> The reason is simple: this issue does not matter to the vast majority of our users.
I can't speak to majority vs minority because I don't have numbers, but
there are clearly (based on number of complaints I've seen) many people
to whom this issue matters, who have Firebug installed, and who don't
realize that _Firebug_ is what's causing their problem.
> To anticipate a counter-argument along the lines of "we're protecting
> users", it's not true. Users may choose to pay 500ms start up time, or
> 5s or 50s for that matter, to use a extension that provides some
> feature
The problem is that users are not consciously choosing to do that,
because they don't realize that the extension is what's causing the
startup slowdown.
> Of course users may 'blame' Firefox for the extra 500ms, meaning they
> did not make an informed choice to pay the 500ms.
Exactly.
> That issue can be addressed by informing users.
We don't have the infrastructure to do that for existing extensions,
obviously (because we integrate extensions much more tightly than IE
does, for example).
Are you going to inform users on the Firebug download or update pages?
-Boris
johnjbarton
> On 4/2/11 8:09 PM, johnjbarton wrote:
>> Startup time is now sub-second, so 75% would be easily crossed by an
>> addon and yet be entirely reasonable for users.
>
> Practice shows that users don't consider anything over about 200ms
> reasonable. They may even _tell_ you that they consider it reasonable,
> but then will complain about things being slow.
I think the practice you cite is not applicable to all user communities.
We have had >4000 bug reports on Firebug and >20000 newsgroup posting.
No one ever complained about startup time. On the other hand we have
lots and lots and lots of complaint about other issues. For the sake of
our users we should concentrate on the issues that matter to them.
>
> So I think we're pretty serious about the 75% startup time overhead
> metric. I happen to think it's pretty generous, myself.
Why? It's not important to Firebug users so what makes it "pretty
serious"? This is really just an arbitrary goal Firefox set for itself
when considering some kind of average user who opens Firefox often. A
very large number of users open Firefox rarely and that includes the
vast majority of Firebug users. You're just creating a fight where none
needs to happen. Let's concentrate on issues that really affect users,
things like memory footprint (a huge issue for Firebug users), support
for JavaScript debugging (spotty and horrible), puzzling error messages
without line numbers, support for debugging Workers, support for
multiprocess and mobile debugging, etc.
I don't really know where the startup time fixation came from, I guess
from seeing Google Chrome come up fast. But hey, the Firefox team did a
great job, FF4 comes up really fast for 400-3 million users. Instead of
creating a huge fuss over <1%, step back and look at the big picture. Is
that 1% the most important play? Or should the priority list be re-examined?
And in the context of governance, idiosyncratic Firefox goals should not
determine policies.
jjb
Boris Zbarsky
> I think the practice you cite is not applicable to all user communities.
> We have had >4000 bug reports on Firebug and >20000 newsgroup posting.
> No one ever complained about startup time. On the other hand we have
> lots and lots and lots of complaint about other issues.
See my other post: the issues they complain are clearly _Firebug_
issues, whereas they think the startup time is a _Firefox_ issue.
> For the sake of our users we should concentrate on the issues that matter to them.
By all means, if you're measuring those correctly.
> It's not important to Firebug users
I believe this premise is false....
> I don't really know where the startup time fixation came from
It's just about our #1 user complaint. I don't mean bug report. I mean
_complaint_, from the vast majority of people, who do not file bugs.
> Instead of creating a huge fuss over <1%, step back and look at the big picture. Is
> that 1% the most important play? Or should the priority list be
> re-examined?
One problem here is that a large majority of the early adopter audience,
who are the ones doing browser reviews and forming opinions
semi-automatically install Firebug "just in case" without even thinking
about whether they need it. And then they write articles about slow
_Firefox_ startup. And normal users read those and make their browser
decisions based on that.
So that 1% can really matter, because if users think we're slow to the
point of not even trying us, then it doesn't matter that we're fast.
(And that's ignoring priming effects in cases when they do try us, etc.)
-Boris
johnjbarton
> On 4/5/11 7:18 AM, johnjbarton wrote:
>> To take a concrete example, consider "adds more than 75% to start-up
>> time". Firebug already exceeds this bar and yet we have not a single
>> bug report or user complaint about startup time.
>
> But _we_ have had complaints about that. And when we suggest that the
> user turn off Firebug they try it, get very happy and say "Oh, is _that_
> what was making things slow?"
News to me. These must be users who close Firefox a lot. We can't make
everyone happy as you know. There will always be problems.
>
>> The reason is simple: this issue does not matter to the vast majority
>> of our users.
>
> I can't speak to majority vs minority because I don't have numbers, but
> there are clearly (based on number of complaints I've seen) many people
> to whom this issue matters, who have Firebug installed, and who don't
> realize that _Firebug_ is what's causing their problem.
Well I do have the numbers, and the numbers say other things are important.
>
>> To anticipate a counter-argument along the lines of "we're protecting
>> users", it's not true. Users may choose to pay 500ms start up time, or
>> 5s or 50s for that matter, to use a extension that provides some
>> feature
>
> The problem is that users are not consciously choosing to do that,
> because they don't realize that the extension is what's causing the
> startup slowdown.
So inform them.
>
>> Of course users may 'blame' Firefox for the extra 500ms, meaning they
>> did not make an informed choice to pay the 500ms.
>
> Exactly.
>
>> That issue can be addressed by informing users.
>
> We don't have the infrastructure to do that for existing extensions,
> obviously (because we integrate extensions much more tightly than IE
> does, for example).
Why obviously? Measuring the major part of the startup time for most
extensions that you would consider a problem (overlays with lots of
script tags and pre-load computation), seems really easy.
>
> Are you going to inform users on the Firebug download or update pages?
We have made a huge investment of time devs and users to insure we
minimize performance impact. We have numerous controls and lots of
documentation for these issues. We haven't bother about startup time
because not a single user has ever complained about it.
We have very limited resources, some things have to be left out.
jjb
>
> -Boris
johnjbarton
> On 4/5/11 9:03 AM, johnjbarton wrote:
>> I think the practice you cite is not applicable to all user communities.
>> We have had >4000 bug reports on Firebug and >20000 newsgroup posting.
>> No one ever complained about startup time. On the other hand we have
>> lots and lots and lots of complaint about other issues.
>
> See my other post: the issues they complain are clearly _Firebug_
> issues, whereas they think the startup time is a _Firefox_ issue.
>
>> For the sake of our users we should concentrate on the issues that
>> matter to them.
>
> By all means, if you're measuring those correctly.
For Firebug users, based on actually countable bug reports and
non-bug-reports, the #1 complaint is memory footprint.
>
>> It's not important to Firebug users
>
> I believe this premise is false....
Based on what?
>
>> I don't really know where the startup time fixation came from
>
> It's just about our #1 user complaint. I don't mean bug report. I mean
> _complaint_, from the vast majority of people, who do not file bugs.
For Firebug users, based on actually countable bug reports and
non-bug-reports, the #1 complaint is memory footprint.
>
>> Instead of creating a huge fuss over <1%, step back and look at the
>> big picture. Is
>> that 1% the most important play? Or should the priority list be
>> re-examined?
>
> One problem here is that a large majority of the early adopter audience,
> who are the ones doing browser reviews and forming opinions
> semi-automatically install Firebug "just in case" without even thinking
> about whether they need it. And then they write articles about slow
> _Firefox_ startup. And normal users read those and make their browser
> decisions based on that.
>
> So that 1% can really matter, because if users think we're slow to the
> point of not even trying us, then it doesn't matter that we're fast.
> (And that's ignoring priming effects in cases when they do try us, etc.)
Thanks, that is exactly the kind of thing I thought was behind this
policy item: an issue important to Firefox marketing rather than really
about users.
As an addon developer I am very keen to see Firefox be both successful
and to be seen as successful. Bring this scenario to our attention,
convince us, let's be on the same team.
Adopting a policy of blocklisting addons because reviewers might install
them and write bad reviews might help Firefox get a couple of better
reviews, but it will destroy the addon development community. It's not
the appropriate tool for the problem you have.
jjb
>
> -Boris
Boris Zbarsky
>> I believe this premise is false....
>
> Based on what?
Based on complaints and bug reports _we_ have received.
>> It's just about our #1 user complaint. I don't mean bug report. I mean
>> _complaint_, from the vast majority of people, who do not file bugs.
>
> For Firebug users, based on actually countable bug reports and
> non-bug-reports, the #1 complaint is memory footprint.
The #1 complaint they report to _you_.
>> So that 1% can really matter, because if users think we're slow to the
>> point of not even trying us, then it doesn't matter that we're fast.
>> (And that's ignoring priming effects in cases when they do try us, etc.)
>
> Thanks, that is exactly the kind of thing I thought was behind this
> policy item: an issue important to Firefox marketing rather than really
> about users.
It's really about users. There are a _lot_ of people with Firebug
installed who complain about startup times as a result. Those are
unhappy users. We want to make them happy.
There's the separate issue that their unhappiness makes them badmouth
the product, so the fact that they're as small fraction of users doesn't
mean they can't have a big impact. But that's the case no matter what;
we've made performance improvements that affect other small user
demographics for similar reasons.
> Adopting a policy of blocklisting addons because reviewers might install
> them and write bad reviews might help Firefox get a couple of better
> reviews, but it will destroy the addon development community. It's not
> the appropriate tool for the problem you have.
All we're really doing here is holding our add-on developers to the same
performance standards we hold ourselves to, albeit with some relaxation
of the acceptable range of slowdown. The relaxation is there because we
recognize that adding anything at all will result in _some_ slowdown and
because we recognize that development resources are limited.
The goal is to have users having a good experience. As side-effect of
this is that they will promote the browser. This sort of goodwill-based
viral effect is what we live or die by. You may not care about it, but
that's your prerogative, of course.
-Boris
Robert Kaiser
> The "10,000 or more in a week" crash threshhold seems like it
> doesn't apply well to betas. Maybe this could be phrased in terms
> of crashes / ADU / day or crashes / ADU / week? (I'm not sure what
> the right number is -- since I'm having trouble figuring out what
> the crashes / ADU number we report actually means.)
I agree that we should put that differently, but I don't think we
actually need to put _any_ number on it. Can we just make it "Causing
high crash volume" and make the CrashKill team determine what qualifies,
or something like that?
Robert Kaiser
--
Note that any statements of mine - no matter how passionate - are never
meant to be offensive but very often as food for thought or possible
arguments that we as a community needs answers to. And most of the time,
I even appreciate irony and fun! :)
Robert Kaiser
> I've been working with Kev Needham, Christian Legnitto, Mike Morgan, and
> others that regularly deal with blocklist issues on a policy draft and
> am ready to propose it to replace the current blocklist policy.
This sounds good to me, though I wonder about vendor/developer request
not being a reason to block. What's the reasoning behind that?
> Additionally, Joe Drew has written a draft policy for the blocking of
> graphics drivers, a new feature in Firefox 4.
There, it would be good if we had ways to engage with the driver
vendors/developers, esp. those of high market share and those who
develop open drivers, to inform them of the problems we're seeing and
perhaps even find ways to fix or avoid them in the future. But I guess
that's an additional step, even if it's somewhat connected to this.
Robert Kaiser
> I object to this item:
>
> Severe performance impact (adds more than 75% to start-up time)
>
> Startup time is now sub-second, so 75% would be easily crossed by an
> addon and yet be entirely reasonable for users.
Adding _anything_ to startup time is something we really badly need to
avoid in the current competitive market of webapp runtimes (commonly
called "browsers") and adding significantly to startup time is
absolutely unacceptable.
There are a lot of ways to load things lazily and not impact startup time.
For debugging, it's OK to slow down startup when you're actually
debugging the application, but no add-on should have such an impact just
by being installed. Such slowed down debug startups should _only_ happen
when a user has specifically requested to start up this instance of the
application this way, by pref or commandline switch or whatever, but it
should _never_ happen by default.
We are getting a real lot of bad user feedback because of slow startup,
and often discover that in those cases some third party software is
actually at fault. We cannot continue to tolerate that, IMHO, and so I'm
cheering for this coming to blocklist criteria and warnings on add-ons
which are increasing startup time being added to AMO as well in the future.
Boris Zbarsky
> them and write bad reviews might help Firefox get a couple of better
> reviews, but it will destroy the addon development community.
I have to ask a question....
Does Firebug actually slow down startup by >75%? If so, why? Other
than the menu item, what do you load at startup? JS components?
-Boris
johnjbarton
> On 4/5/11 9:34 AM, johnjbarton wrote:
>> Adopting a policy of blocklisting addons because reviewers might install
>> them and write bad reviews might help Firefox get a couple of better
>> reviews, but it will destroy the addon development community.
>
> I have to ask a question....
>
> Does Firebug actually slow down startup by >75%?
I don't know, that is just what AMO says.
> If so, why? Other than
> the menu item, what do you load at startup? JS components?
Well as I've said it's not something we've ever really looked at in detail.
We load ~50kloc JavaScript, some XUL overlays for the outer Firebug UI,
menu items, and buttons, then run our initializers. Mostly this is
standard stuff based on the extension model. None of the analysis code
runs on load; the panel content it built on demand. We have no
components; we have about 10 Cu.import jsm modules and about 10 AMD
modules (our new direction for 1.8).
As far as I am aware, there is no simple way to delay the startup or
change over to eg Cu.import. Our code is 'modular', meaning that the
outer function must run to register the modules for initialization and
these modules are per XUL window. Placing them a Cu.import would break
all of that.
We might be able to gain some by splitting the initialization into
stuff related to menus and the rest. But fundamentally this will be a
change in user experience: we currently are active during the load. How
can we know if the user wants a page debugged that is on the session
restore list?
jjb
johnjbarton
> johnjbarton schrieb:
>> I object to this item:
>>
>> Severe performance impact (adds more than 75% to start-up time)
>>
>> Startup time is now sub-second, so 75% would be easily crossed by an
>> addon and yet be entirely reasonable for users.
>
> Adding _anything_ to startup time is something we really badly need to
> avoid in the current competitive market of webapp runtimes (commonly
> called "browsers") and adding significantly to startup time is
> absolutely unacceptable.
The real world says that your competitive problem is memory. I really
think you are over stating the startup problem. Firebug is less than 1%
of your user base.
>
> There are a lot of ways to load things lazily and not impact startup time.
yes, of course this is true. We'd do it immediately if it was easy. I
don't think it is easy.
>
> For debugging, it's OK to slow down startup when you're actually
> debugging the application, but no add-on should have such an impact just
> by being installed.
But if the user wants to debug a page and that page is loaded when
Firefox starts, what shall we do?
> Such slowed down debug startups should _only_ happen
> when a user has specifically requested to start up this instance of the
> application this way, by pref or commandline switch or whatever, but it
> should _never_ happen by default.
Users install addons, we never force them to do that.
>
> We are getting a real lot of bad user feedback because of slow startup,
> and often discover that in those cases some third party software is
> actually at fault. We cannot continue to tolerate that, IMHO, and so I'm
> cheering for this coming to blocklist criteria and warnings on add-ons
> which are increasing startup time being added to AMO as well in the future.
I hope you will reconsider this attitude and approach this as a group
effort rather than painting your addons as third parties to be blocked
and punished.
>
> Robert Kaiser
>
Asa Dotzler
> And in the context of governance, idiosyncratic Firefox goals should not
> determine policies.
We already have policies about Firefox performance requirements. This
proposal suggests extending some of those policies to our add-ons. It is
not "idiosyncratic Firefox goals" that we want and require our
developers to not regress performance (policies going back a decaded)
and it is not at all unreasonable to create policy that extends this to
add-ons.
- A
Max Kanat-Alexander
> These must be users who close Firefox a lot.
Like, say, me.
Also, most users shut down and start their computer at least every day.
I have been a Firebug user and I expect my browser to start quickly.
> We haven't bother about startup time because not a single user has ever complained about it.
Why would they ever complain to you? How could they possibly know it's
a Firebug issue?
-Max
Asa Dotzler
>
> The real world says that your competitive problem is memory. I really
> think you are over stating the startup problem. Firebug is less than 1%
> of your user base.
John, you are simply wrong here. We have the data. Start-up time and
stability far far far outweight memory usage in large sample feedback we
get.
Your Firebug users are a tiny fraction of Firefox users and I'm not
going to challenge your data on what they report to you, but your
insistence that we're focused on the wrong problem for our users isn't
supported by what our users tell us.
- A
Asa Dotzler
> As a matter of policy, meaning a way of making decisions based on
> principles rather than whim, this item has no place on the list.
> Performance is a choice, it is like tabs-on-top, it is not like
> security, crashes, harm, or unintended side effects.
I've said this above but I'll say it again. We HAVE and HAVE HAD FOR A
DECADE policies on performance. We don't let our developers regress
start-up, new window, page load, etc. performance even 1% much less 75%.
We are absolutely within our normal ways of working when we create
policies about performance. You may not like that, but the rest of the
Mozilla community has supported it since long before you came to the
project.
- A
Boris Zbarsky
>> Does Firebug actually slow down startup by >75%?
>
> I don't know, that is just what AMO says.
Where? All I've seen so far in this thread is a proposal that addons
that slow down startup by >75% be blocklisted. Firebug wasn't mentioned
until you brought it up. Am I missing something?
-Boris
Ken Saunders
Asa Dotzler
johnjbarton
> On 4/5/2011 7:18 AM, johnjbarton wrote:
>
> > As a matter of policy, meaning a way of making decisions based on
> > principles rather than whim, this item has no place on the list.
> > Performance is a choice, it is like tabs-on-top, it is not like
> > security, crashes, harm, or unintended side effects.
>
> I've said this above but I'll say it again. We HAVE and HAVE HAD FOR A
> DECADE policies on performance. We don't let our developers regress
Please don't yell it's not necessary.
> start-up, new window, page load, etc. performance even 1% much less 75%.
> We are absolutely within our normal ways of working when we create
> policies about performance. You may not like that, but the rest of the
> Mozilla community has supported it since long before you came to the
> project.
I've worked on Firebug for four years. You've been keeping
secrets ;-).
jjb
>
> - A
johnjbarton
Ok I take it back. Let's agree then: Firebug is a tiny fraction of
Firefox users and the needs of Firebug users do not overlap 100% with
Firefox users.
So using performance limitations based on the entire Firefox
population for the Firebug users does not make sense right?
jjb
>
> - A
johnjbarton
That is an important issue we can solve with technology. It's not an
argument to blocklist addons.
jjb
>
> -Max
Max Kanat-Alexander
> That is an important issue we can solve with technology. It's not an
> argument to blocklist addons.
Okay, fair enough. As long as we all agree that it's an important
issue. :-)
Perhaps a better policy would be "blocklist addons that cause
significant performance degradations and don't fix these after a long
time (say, one year)".
-Max
johnjbarton
Sorry, I would still disagree with this. Performance is a just like
other variables with important tradeoffs to be made by users and addon
devs, not dictated by Firefox.
Imagine if Linus or Microsoft had such a policy and announced that
Firefox would be banned from the operating system unless they did
things to make Linux or Windows look better? Tell the truth: would
your first reaction be "Yes Sir, right away Sir!". I don't think so.
jjb
>
> -Max
Asa Dotzler
I think that "the entire Firefox population" as best we can measure it,
cares about start-up time and so I do not think that a softblock
(basically a notification of the problem with the ability to continue on
using the extension) is at all unreasonable, even in the Firebug case.
- A
Asa Dotzler
Yes, this is an issue we can solve with technology. The technology is a
soft block and it let those users know that it is Firebug that is
causing their performance problems. They are free to accept that
performance penalty and continue using Firebug but they will have been
notified. I think this is a great technology solution to this issue.
- A
Gijs Kruitbosch
>How can we know if the
> user wants a page debugged that is on the session restore list?
>
> jjb
Posit: nobody ever wants that.
Can you think of a scenario where this is desired behaviour? You just started
your browser, it loads 20 tabs from the last time you closed it, and you
immediately want to start to debug some of them? Of course, there may be rare
issues where debugging a page crashed the browser, and the user would like to
resume where they left off, but (1) there's a bigger issue in such a case :-),
and (2) in this case it's probably better not to resume debugging straight off,
as it's quite likely to crash *again*... I can't think of any other cases where
users want debugging off-the-bat.
If you think there are rare cases, add a hidden pref for loading stuff on
startup if you want. Otherwise, lazy initialization will be a huge boon to the
users.
Separately, when you say "we've never really looked into it", I'm genuinely
scared. Back in 2006, I spent some time optimizing Venkman startup as part of my
Google Summer of Code project (in which I also worked on Firebug a bit). It took
me very little time and effort to shave off a rough 50% from the time it took to
open the Venkman window after startup, mainly by not enumerating all the loaded
scripts and processing their lines' executable-ness (linetopc and the reverse
are slow, as you probably know).
Of course, Firebug is a different beast, and at the time startup time for the
browser was not such a hot topic. Different measures will undoubtedly be
necessary here. Regardless, I will be frank and admit that I find it a bit
presumptuous of you to go yelling that all this is unfair or unnecessary while
you've never even looked into the matter. Boris is right that speed is a very
big concern of many users, even if you don't see it in their bugreports (and
anyway, filing good bug reports about speed issues is Hard, as others pointed
out - going from "Firefox is slow" to "Firefox is slow because of ..." is
impossible for most users).
Finally, as a completely arbitrary data point, as a web developer (my day job),
I run two separate Firefox profiles. One for browsing, reading documentation,
etc., and one with Firebug. Why? Because I can feel how much slower Firefox is
with Firebug installed, to start up as well as to browse.
Gijs
Gijs Kruitbosch
>How can we know if the
> user wants a page debugged that is on the session restore list?
>
> jjb
Posit: nobody ever wants that.
timeless
> Imagine if Linus or Microsoft had such a policy and announced that
> Firefox would be banned from the operating system unless they did
> things to make Linux or Windows look better?
Actually, Microsoft does do this.
verifier.exe - http://support.microsoft.com/kb/244617
Microsoft requires drivers to be signed (WDM) and getting a signature
requires going through the Windows Logo program / WinQAL.
The requirements include using driver verifier which includes things
like "must not hang/deadlock system"
The latest version of their docs seems to require an account, but you
can review the old requirements as they apply to Windows XP/2003
Server. Please trust that Microsoft generally *adds* requirements, and
is unlikely to remove requirements, especially verifier.
http://download.microsoft.com/download/3/2/c/32cb69db-71e3-447e-a91b-4070cdc4548a/WLP22-10_a.doc
• Driver Verifier. For each Windows XP or Windows Server 2003 family
driver, no errors can occur under the Driver Verifier facility
provided with the operating system.
Poorly written kernel-mode drivers have the potential to cause the
system to become unstable or stop working. Therefore, it is critical
that all kernel-mode drivers be thoroughly tested to minimize this
risk. For information about using Driver Verifier and diagnosing
driver problems, see http://go.microsoft.com/fwlink/?LinkId=36677.
Microsoft hasn't switched to doing this for Userland applications.
However of note, Apple to some extent with its App Store for 10.6.7+
could choose or perhaps already does enforce performance requirements
on Userland applications.
timeless
<gijskru...@gmail.com> wrote:
> Posit: nobody ever wants that.
Indeed.
> Can you think of a scenario where this is desired behaviour? You just
> started your browser, it loads 20 tabs from the last time you closed it, and
> you immediately want to start to debug some of them?
IME one of the pages I'm on in my session restore will actually
trigger my crash, and while I'd like to understand why, it's
incredibly frustrating hitting the restore button and watching firefox
die *again*. So no, even when you think you want it, you really don't.
> Of course, there may be
> rare issues where debugging a page crashed the browser, and the user would
> like to resume where they left off, but (1) there's a bigger issue in such a
> case :-), and (2) in this case it's probably better not to resume debugging
> straight off, as it's quite likely to crash *again*... I can't think of any
> other cases where users want debugging off-the-bat.
Indeed. Having more or less run into this process about a dozen times
last week while doing work (Venkman hit a bug in some layout code).
And another dozen times this week with some of my code just the act of
opening the crash report pages triggered crashes. Getting right back
to the crash that triggered your restart is not really what you want,
even if you do want to eventually chase it.
johnjbarton
In making this decision you have to balance the importance of an
occasional bad review against the ill will you will create in your
addon community. It's your community to destroy or grow. I think it's
not a good trade off, especially since there are much more productive
and pleasant solutions.
jjb
>
> - A
johnjbarton
> On 05/04/2011 20:32 PM, johnjbarton wrote:
>
> >How can we know if the
> > user wants a page debugged that is on the session restore list?
>
> > jjb
>
> Posit: nobody ever wants that.
>
> Can you think of a scenario where this is desired behaviour? You just started
> your browser, it loads 20 tabs from the last time you closed it, and you
> immediately want to start to debug some of them? Of course, there may be rare
> issues where debugging a page crashed the browser, and the user would like to
> resume where they left off, but (1) there's a bigger issue in such a case :-),
> and (2) in this case it's probably better not to resume debugging straight off,
> as it's quite likely to crash *again*... I can't think of any other cases where
> users want debugging off-the-bat.
I can see why you might think this way based on work with C++ code.
But Firebug users don't face these kinds of issues.
Based on conversations with users, a common use model -- among users
who don't normally have Firefox open and thus are affected by the
startup time -- to check on one of their sites. So yes, I expect debug-
on-startup to be the common case for such users.
>
> If you think there are rare cases, add a hidden pref for loading stuff on
> startup if you want. Otherwise, lazy initialization will be a huge boon to the
> users.
I agree that lazy initialization should be a goal and that it would
help many Firebug users. However it just not a simple matter to
implement.
>
> Separately, when you say "we've never really looked into it", I'm genuinely
> scared. Back in 2006, I spent some time optimizing Venkman startup as part of my
> Google Summer of Code project (in which I also worked on Firebug a bit). It took
> me very little time and effort to shave off a rough 50% from the time it took to
> open the Venkman window after startup, mainly by not enumerating all the loaded
> scripts and processing their lines' executable-ness (linetopc and the reverse
> are slow, as you probably know).
I never looked into it because not a single user ever complained.
>
> Of course, Firebug is a different beast, and at the time startup time for the
> browser was not such a hot topic. Different measures will undoubtedly be
> necessary here. Regardless, I will be frank and admit that I find it a bit
> presumptuous of you to go yelling that all this is unfair or unnecessary while
> you've never even looked into the matter. Boris is right that speed is a very
> big concern of many users, even if you don't see it in their bugreports (and
> anyway, filing good bug reports about speed issues is Hard, as others pointed
> out - going from "Firefox is slow" to "Firefox is slow because of ..." is
> impossible for most users).
I never looked in to the matter because no user ever complained about
it and no one from Firefox explained why they thought it was
important.
Now that I have heard about this issue from the Firefox side I'll look
into it more. Well after I let the poor way this affair was handled
wear off.
>
> Finally, as a completely arbitrary data point, as a web developer (my day job),
> I run two separate Firefox profiles. One for browsing, reading documentation,
> etc., and one with Firebug. Why? Because I can feel how much slower Firefox is
> with Firebug installed, to start up as well as to browse.
A great strategy employed by many users and more evidence that this
issue just is not as critical as other issues on the blocklist threat.
>
> Gijs
Asa Dotzler
It's not the occasional bad review that I'm worried about. It's the huge
number (tens of millions, maybe more) of users who are having a bad
experience with Firefox startup because they have extensions installed
that are hurting their startup and they don't know they can do anything
about it.
I do think we need to balance that against the happiness of our add-on
community.
I think it's a fine balance to trade 1) making a few add-on authors with
particularly egregious extensions feel bad because their extensions
essentially come with an extra warning to end users about performance
impact, for 2) the user empowerment for hundreds of millions of Firefox
users who will get to know if an extension regresses their performance
significantly and be be able to respond how ever they see fit.
- A
Ken Saunders
> I think it's a fine balance to trade 1) making a few add-on authors with
> particularly egregious extensions feel bad because their extensions
> essentially come with an extra warning to end users about performance
> impact, for 2) the user empowerment for hundreds of millions of Firefox
> users who will get to know if an extension regresses their performance
> significantly and be be able to respond how ever they see fit.
>
> - A
That's totally fair imo because most users aren't aware of the impact
that add-ons have on performance and they do blame Firefox for it, but
is blocklisting an add-on for performance issues fair? Not major
performance issues of course.
And the exact areas of performance that an add-on may have an affect
on should be noted to help users make a more informed decision. If
it's just said that an add-on will make Firefox 75% slower and the
issue is just startup time and not overall, then that should be made
clear. A user might not have a problem with that as opposed to
something like more memory consumption.
Ken
Michael Lefevre
> I think it's a fine balance to trade 1) making a few add-on authors with
> particularly egregious extensions feel bad because their extensions
> essentially come with an extra warning to end users about performance
> impact, for 2) the user empowerment for hundreds of millions of Firefox
> users who will get to know if an extension regresses their performance
> significantly and be be able to respond how ever they see fit.
This wording, and the wording around the web graph, is too vague though.
As you said elsewhere in the thread, Mozilla itself has a whole set of
tests measuring a whole set of different things. But now "add-on
performance" is measured by the single metric of (AIUI) cold start up in
some unrealistic (clean profile) test environment.
If some tech publication came along tomorrow and said "Firefox 4 is 20
times slower than IE 9" and based their performance measures only on
cold start up, would you think that was fair?
If an add-on was, say, causing a delay of 30 seconds to every page load,
but only delaying start up by 1%, would you be happy to see that add-on
rated as a top performer?
It's also not true to say that none of the Firefox measures are ever
allowed to regress. At least a couple of things regressed for Firefox 4
and those were considered acceptable trade-offs for other gains.
Michael
Asa Dotzler
This is starting to veer off topic, but I can tell you with absolute
certainty that if any change landing in Firefox caused a start-up
regression of anywhere close to 75%, that change would be backed out
immediately. We bust our ass for single digit percentage point gains on
start-up. A 10% gain would make a hero out of the developer who came up
with it. Seeing a few years worth of incremental start-up wins being
wiped out by an extension and being blamed on Firefox is unacceptable.
- A
johnjbarton
http://twitter.com/#!/search/firebug
you can see the effect of this campaign: Firebug users are switching
to Chrome dev tools. Now we can predict success: soon you won't have
to worry about bad press from Firefox tests with Firebug installed.
jjb
Ken Saunders
I'd say that the comments in the following article provides a good
idea of what current Firebug users think about it being on the Slow
Performing Add-ons list, and so I wouldn't be too concerned about
current Firebug users abandoning Firefox, but it may slow down
adoption of Firebug in the future.
I've always found the comments from a large number of consumers/end
users to be more useful and a good barometer overall than the opinions
from a single writer anyway regardless of who the author may be.
Mozilla Hangs Slow Firefox Add-ons on a Wall of Shame
http://mashable.com/2011/04/06/slow-firefox-add-ons/
While I personally don't see the big deal, and have never had any
issues with startup time, it is a big complaint amongst users (perhaps
because of the media), so in the bigger scheme of things Mozilla has
to do something. I'm as big of a defender and advocate for add-on
developers as I am of Firefox so I'd really like to see this all
worked out.
Perhaps Mozilla can change the messaging a bit and like I mentioned
before, provide specifics on the performance issues (increases startup
time, increases memory consumption, slows down page loading, etc), and
developers can educate their users (and potential ones) on how and why
their add-ons may affect performance and what they can do about it
(setting up developmental profiles, disabling components, custom
installs, etc ).
Though they are a minority, there are some Firefox end users with
special needs that are having to deal with having some veteran Firefox
theme developers and long time Mozilla contributors retiring,
quitting, or just giving up due to some bad blood (and other things)
between themselves and Mozilla. Those end users will either choose not
to upgrade to 4.0, or just try and live with things as they.
I'd hate to see the same thing happen with extension developers too.
It's the end user that loses.
I sincerely hope that things can be worked out.
Ken
Justin Scott
Adding this to the blocklist policy does not mean we would blocklist in
every case. We'd look at each individual add-on and work with the
developer to improve start-up time and only soft-block if the impact is
excessive for the add-on's functionality.
While it is debatable whether Firebug's impact is excessive for its
functionality, my opinion is that as a deeply-integrated development
tool, we should not block it as long as users are made aware of the
impact so they can make an educated decision to install it.
As discussed in private email with you prior to the performance campaign
launch, in Firebug's case we are satisfied with the warnings and will
not pursue other options (removing from AMO, blocklisting, etc.)
Please don't use this blocklist thread to attack the performance
campaign in general. There really are add-ons that add a minute or 30
seconds to start-up time and we need to block them if the developers
won't fix them. We've added 75% to the policy because we have to draw
the line somewhere and a single add-on that needs 3/4 the time to start
that Firefox itself does is excessive in most cases. It's unfortunate
that Firebug is at that limit, but this is much bigger than just Firebug.
Justin
johnjbarton wrote:
> On 4/2/2011 5:32 PM, Justin Scott wrote:
>> Hi,
>>
>> The current blocklist policy (https://wiki.mozilla.org/Blocklisting) has
>> been in place since the feature was introduced in Firefox 2, and since
>> then we've blocked quite a few add-ons and have a much better idea of
>> the circumstances under which we want to block add-ons and plugins.
>>
>> I'd like to revamp this policy to provide more guidance on when we
>> should block and establish more specific criteria for blocking so that
>> it's easier to make decisions on whether to block something or not.
>>
>> I've been working with Kev Needham, Christian Legnitto, Mike Morgan, and
>> others that regularly deal with blocklist issues on a policy draft and
>> am ready to propose it to replace the current blocklist policy.
>>
>> Additionally, Joe Drew has written a draft policy for the blocking of
>> graphics drivers, a new feature in Firefox 4.
>>
>> Please take a look at the drafts below and share any feedback.
>>
>> https://wiki.mozilla.org/User:Fligtar/Blocklist_Draft
>
> I object to this item:
>
> Severe performance impact (adds more than 75% to start-up time)
>
> Startup time is now sub-second, so 75% would be easily crossed by an
> addon and yet be entirely reasonable for users.
>
> More generally, some addons have performance impact and there is no
> practical alternative or technical solutions. Examples from Firebug
> include memory overhead ("1.7.0 consumed 2.5GB over night" to quote a
> user), page rendering overhead (DOM mutation listeners), script overhead
> and so on.
>
> To me there should be no bar on non-malicious performance, it's between
> the addon dev and the user.
>
> jjb
>
>
>> https://wiki.mozilla.org/User:JoeDrew/GFXBlocklistDraft
>>
>> Thanks,
>> Justin
>
Justin Scott
johnjbarton
> John,
>
> Adding this to the blocklist policy does not mean we would blocklist in
> every case. We'd look at each individual add-on and work with the
> developer to improve start-up time and only soft-block if the impact is
> excessive for the add-on's functionality.
While I understand that this seems like a measured and reasonable
approach to you, it is not. The purpose of a policy is to clarify in
writing otherwise arbitrary boundaries so that addon authors know what
to expect. A policy which amounts to "we might destroy your work but
we might not, it depends" really isn't a policy. I don't know what to
expect, so I have to plan for the worst case.
That is why I am urging you to consider performance as off your list.
Performance is very important to you, that's great. But so is your
color scheme, user interaction model, and so on. There are educational
and instrumentation approaches which can help users make informed
choices in these cases. Color scheme is easy to detect, startup time
not so much but it can be fixed.
As a simple compromise, just leave this item off until you can
determine if something simple performance measurements and UI help
users.
>
> While it is debatable whether Firebug's impact is excessive for its
> functionality, my opinion is that as a deeply-integrated development
> tool, we should not block it as long as users are made aware of the
> impact so they can make an educated decision to install it.
>
> As discussed in private email with you prior to the performance campaign
> launch, in Firebug's case we are satisfied with the warnings and will
> not pursue other options (removing from AMO, blocklisting, etc.)
>
> Please don't use this blocklist thread to attack the performance
> campaign in general.
That wasn't me, somehow my example got out and ran wild ;-).
> There really are add-ons that add a minute or 30
> seconds to start-up time and we need to block them if the developers
> won't fix them.
Want to, perhaps. Need to: we debate.
> We've added 75% to the policy because we have to draw
> the line somewhere and a single add-on that needs 3/4 the time to start
> that Firefox itself does is excessive in most cases. It's unfortunate
> that Firebug is at that limit, but this is much bigger than just Firebug.
Exactly why I urge you to remove this item from your list.
jjb
Shawn Wilsher
> This wording, and the wording around the web graph, is too vague though.
> As you said elsewhere in the thread, Mozilla itself has a whole set of
> tests measuring a whole set of different things. But now "add-on
> performance" is measured by the single metric of (AIUI) cold start up in
> some unrealistic (clean profile) test environment.
determine add-on performance. This is simply the *first* test we are
running and starting to report.
Cheers,
Shawn
Michael Lefevre
I wasn't actually making that assumption myself anyway, but the media
coverage of the campaign seems to have done so. The wording used around
the results did not imply that this was anything preliminary or
incomplete. You have done performance testing, reported results and
they've been reported on various tech sites and caused a bunch of
discussion on forums, twitter and wherever else.
Reporting additional results later or refining the implementation of the
policy is nice, but I doubt it's going to get as much attention as that
original graph... but I hope I'm wrong...
Michael
Asa Dotzler
> Reporting additional results later or refining the implementation of the
> policy is nice, but I doubt it's going to get as much attention as that
> original graph... but I hope I'm wrong...
>
> Michael
I'm not quite sure what you're getting at here, but I don't think
additional tests make extensions less likely to get blocklisted. Things
only get stricter if, for example, we come up with good pageload time
tests that show extensions slowing that down, or a good new window time
test that show extensions slowing that down or good memory tests showing
extensions pushing Firefox into unreasonable amounts of RAM usage.
- A
Justin Scott
(http://blog.mozilla.com/addons/2011/04/01/improving-add-on-performance/) is
clear that this is only the first metric and a first step, but
unfortunately it seems many people only looked at the results page. The
results page is intentionally worded about performance in general
because all the performance tests will be listed there once they are
available.
I'm looking at changing the wording now since several developers have
mentioned it, but let's keep this thread on topic.
Justin Scott
(http://blog.mozilla.com/addons/2011/04/01/improving-add-on-performance/) is
clear that this is only the first metric and a first step, but
unfortunately it seems many people only looked at the results page. The
results page is intentionally worded about performance in general
because all the performance tests will be listed there once they are
available.
I'm looking at changing the wording now since several developers have
mentioned it, but let's keep this thread on topic.
Siddharth Agarwal
>
> https://wiki.mozilla.org/User:Fligtar/Blocklist_Draft
I definitely like the idea of warning users about slow extensions, but
I'm worried about the wording of the dialog for addons that slow Firefox
down. A slowdown by itself causes neither stability nor security
problems, and I think the dialog needs to express that. For the same
reason, I don't see how the red X in the dialog makes sense.
The way I'd do it is to have each blocklist entry contain a set of
reasons about why it was blocklisted, and to display that to the user. I
don't know how practical that is to implement, though.
I also wonder if popping up a dialog is too intrusive. I think IE9 does
it well with its notification at the bottom.
I think the new policy should only be implemented after this sort of
updated, "softer" UI is in place.
Justin Scott
this more and become more certain in our position that this is the right
thing to do. There is broad consensus in this thread and among those of
us who are responsible for delivering an awesome experience for Firefox
users and who work with the add-ons ecosystem every day that performance
is a critical piece that we can't ignore.
We are putting 75% start-up time as an example in the policy so that
authors know what we expect, but as I said we will use discretion when
enforcing this as we do with all blocklist criteria. As you are against
blocking add-ons that even add 30 seconds or a minute to start-up time,
it's clear that you think performance should not be a factor in this at
all and we'll remain in disagreement on that.
We're going to go ahead with the performance aspect of this draft policy.
Justin