| A summary of ideas to improve add-ons | Nicholas Nethercote | 3/14/12 4:36 PM | Hi,
There's been a lot of discussion about add-ons lately, and lots of ideas suggested on how to improve them. I've attempted to summarize the problems and the ideas in this etherpad: https://etherpad.mozilla.org/M3nBRlRiXF The three problem areas are "Add-ons can hurt performance", "Non-AMO add-ons have no minimal quality standard", "Foreign-installed add-ons can be unwanted". There is some overlap between the first two areas (performance is an aspect of quality) but I felt they were worth separating. Some of the listed ideas are already in place. Of those that are not, I think the following are both worth doing and likely to be uncontroversial: - offer reviews/QA for non-AMO add-ons - provide listings for non-AMO add-ons on AMO (Just because an idea is controversial doesn't mean it's bad. But given how much disagreement there has been about this stuff lately, I thought it worth highlighting these ideas right from the start :) Nick |
| Re: A summary of ideas to improve add-ons | Henri Sivonen | 3/15/12 3:23 AM | I've noticed that @firefox on Twitter has promoted at least one add-on
that you list as having leaks (Lastpass). Maybe we should have some rule of not promoting foreign-installed add-ons or add-ons with known problems or add-ons that replace features (like password manager or safe browsing) that Firefox already has built-in. -- Henri Sivonen hsiv...@iki.fi http://hsivonen.iki.fi/ |
| Re: A summary of ideas to improve add-ons | Michael Lefevre | 3/15/12 4:05 AM | On 15/03/2012 10:23, Henri Sivonen wrote:I know you've used "or", but that starts to look like a way of never promoting any add-ons at all. Lastpass might be foreign-installed as well, but it is also on AMO, fully reviewed and listed as a "featured" addon. Actually I can immediately see several add-ons on the AMO featured list which are also on the "leaking" list, and which replace built-in functions. If users get told too much that add-ons are bad and hurt performance, then they're not going to want any add-ons. Additionally, Lastpass (and similar) have been promoted (I think within Mozilla forums as well as elsewhere) specifically because the built-in feature is lacking in some areas and are no longer being worked on (which gives me an opportunity to complain about bug 355063 - Password manager does not work on script-generated forms - which affects more and more sites). Michael |
| Re: A summary of ideas to improve add-ons | Henri Sivonen | 3/15/12 4:39 AM | On Thu, Mar 15, 2012 at 1:05 PM, Michael LefevreSurely there are non-leaky add-ons on AMO that don't replace built-in functionality. I think leaky add-ons shouldn't be on the features list, either. |
| Re: A summary of ideas to improve add-ons | Justin Lebar | 3/15/12 7:39 AM | >> Lastpass might be foreign-installed as well, but it is also on AMO, fullyAgreed. But leaky add-ons shouldn't be on AMO at all. This is something we've basically agreed on, and we're moving in that direction. All new add-ons are reviewed for leaks; add-ons which leak are not approved for AMO. When we find a leak in an add-on post-review (or in an add-on which was approved before we started testing for leaks), we contact the author and, if the add-on is not fixed in a timely fashion, we downgrade the add-on to preliminary status on AMO. This has been the topic of some heroic work by the add-ons folks. A lot of issues have been fixed, and we're just getting started. Instead of simply downgrading to preliminary, I think we should a) Say on the AMO page that the add-on is known to leak -- the preliminary review warning is not particularly informative [1] -- and b) Notify all users who have the add-on installed that it may be causing problems. There was originally some question as to whether (a) and (b) would be necessary. It might have been that we'd have to downgrade very few add-ons, because most authors would be responsive and fix their bugs. I think we've observed now that this is not the case. Despite plenty of bugs fixed, we've also downgraded some popular add-ons [2]. So I hope we'll get to work on (a) and (b) above. [1] An add-on in preliminary review looks like: https://addons.mozilla.org/en-US/firefox/addon/baow/ [2] Downgraded add-ons that I'm aware of include: https://bugzilla.mozilla.org/show_bug.cgi?id=728528 <-- 250 users. Not a popular add-on. :) https://bugzilla.mozilla.org/show_bug.cgi?id=691102 <-- ~50th most popular AMO add-on, 400,000 users, compatible through FF 11 https://bugzilla.mozilla.org/show_bug.cgi?id=716163 <-- 90th most popular AMO add-on, 150,000 users, compatible through FF 11 |
| Re: A summary of ideas to improve add-ons | Kyle Huey | 3/15/12 7:52 AM | On Thu, Mar 15, 2012 at 3:23 AM, Henri Sivonen <hsiv...@iki.fi> wrote:What's wrong with addons replacing built-in features, if the replacement is at the same level of quality (in terms of leaks/perf/etc)? - Kyle |
| Re: A summary of ideas to improve add-ons | Henri Sivonen | 3/15/12 7:55 AM | It sets users up to grief like what was seen when Google Toolbar went
away and people had their bookmarks there. Having passwords in a 3rd-party password manager is a setup for a situation where user workflow will be badly disrupted if the extension stops working. Outright *promoting* such extensions sets us up for having problems with "too disruptive to fail" add-ons. As for safe browsing replacements, are they really same quality in terms of leaks and perf as the built-in feature? |
| Re: A summary of ideas to improve add-ons | Jorge Villalobos | 3/15/12 9:28 AM | On 3/15/12 4:23 AM, Henri Sivonen wrote:The memory leak hunt is only getting started, so we're still in a discovery phase where many add-on leaks will be discovered. We're only beginning to inform developers how to look for leaks and fix them. While some are the result of bad coding practices, others can be caused by minor oversights (like an undeclared variable). I expect many popular add-ons to have leaks, since they usually have the most complex code, and given that developers haven't been actively looking for them since they don't know how to. We only promote AMO add-ons as far as I know. Depending on the severity of the problem, we already have this policy. We don't promote add-ons that are not fully reviewed, and we work with the Featured Add-ons group to make sure new add-ons being featured aren't problematic. I disagree with this completely. - Jorge |
| Re: A summary of ideas to improve add-ons | Jorge Villalobos | 3/15/12 9:28 AM | On 3/15/12 4:23 AM, Henri Sivonen wrote: > I've noticed that @firefox on Twitter has promoted at least one add-on The memory leak hunt is only getting started, so we're still in a > Maybe we should have some We only promote AMO add-ons as far as I know. > or add-ons with known problems Depending on the severity of the problem, we already have this policy. > or add-ons that replace features (like password manager or |
| Re: A summary of ideas to improve add-ons | Dave Townsend | 3/15/12 12:44 PM | On 03/14/12 16:36, Nicholas Nethercote wrote:We used to do this, I don't recall why we stopped. |
| Re: A summary of ideas to improve add-ons | Michael Lefevre | 3/15/12 3:32 PM | On 15/03/2012 14:55, Henri Sivonen wrote:... >> What's wrong with addons replacing built-in features, if the replacement isMy understanding of this was that there was a benefit to Mozilla in not having to develop and maintain the additional fixes/features that these add-ons provide. If there's a problem, then rather than trying to hide their existence from people, an obvious (and more positive) solution would be to make the fixes and add the features that (some) people want to the built-in facility. Adding a bunch of features and fixes to the download manager, password manager and tabbed browsing could wipe out the popularity of several potentially "too disruptive to fail" add-ons pretty quickly. Michael |
| Re: A summary of ideas to improve add-ons | Nicholas Nethercote | 3/15/12 3:44 PM | On Thu, Mar 15, 2012 at 9:28 AM, Jorge Villalobos <jo...@mozilla.com> wrote:This all sounds fine to me. Can we not get too distracted by the issue of promoted add-ons? I think they are a very minor part of this overall discussion. Nick |
| Re: A summary of ideas to improve add-ons | Nicholas Nethercote | 3/15/12 3:52 PM | On Thu, Mar 15, 2012 at 7:39 AM, Justin Lebar <justi...@gmail.com> wrote:How are preliminarily reviewed add-ons distinguished on AMO? I can see they have an easily-missed bit of text under the big "Add to Firefox" button that says: This add-on has been preliminarily reviewed by Mozilla. Learn More. The "Learn More" link says this: What does it mean if an add-on is "experimental" or "preliminarily reviewed"? Experimental add-ons have been checked by our editors to make sure they don't have security problems, but they may still have bugs or not work properly. Use caution when installing experimental add-ons and uninstall the add-on immediately if you notice problems. which doesn't actually explain what "preliminarily reviewed" means. There's a "Learn more about our review process" link. If you follow that you get to https://addons.mozilla.org/en-US/developers/docs/policies/reviews, which finally explains what it means ("preliminarily reviewed" and "experimental" are basically synonyms, AFAICT). Are preliminarily reviewed add-ons distinguished in any other way on AMO? Nick |
| Re: A summary of ideas to improve add-ons | Jorge Villalobos | 3/15/12 4:02 PM | |
| Re: A summary of ideas to improve add-ons | Jorge Villalobos | 3/15/12 4:07 PM | * You'll see a second warning when you click on the Add to Firefox button.
* On search results, fully reviewed add-ons will always rank higher. * They are never featured, so they won't show up in the most prominent places (except maybe in the most popular rankings, if they are very popular). * The button has a different color (yellow with stripes instead of green). - Jorge |
| Re: A summary of ideas to improve add-ons | Jorge Villalobos | 3/15/12 4:07 PM | On 3/15/12 4:52 PM, Nicholas Nethercote wrote: |
| Re: A summary of ideas to improve add-ons | Jorge Villalobos | 3/15/12 4:10 PM | On 3/15/12 5:07 PM, Jorge Villalobos wrote:Scratch this. That is only true for unreviewed add-ons, not preliminarily reviewed ones. - Jorge |
| Re: A summary of ideas to improve add-ons | Jorge Villalobos | 3/15/12 4:10 PM | On 3/15/12 5:07 PM, Jorge Villalobos wrote: > * You'll see a second warning when you click on the Add to Firefox button. |
| Re: A summary of ideas to improve add-ons | Nicholas Nethercote | 3/15/12 4:18 PM | On Thu, Mar 15, 2012 at 4:02 PM, Jorge Villalobos <jo...@mozilla.com> wrote:Oh, that's interesting, thanks for the link. The key quote seems to be this: "many of the best add-ons that aren’t hosted on AMO did not participate. Instead, add-ons that were already listed on AMO switched to self-hosted to avoid complying with certain review policies." Unintended side-effects! I guess the visibility benefits of being on AMO are high enough that it makes it worthwhile to go through the hurdle of the review process. Nick |
| Re: A summary of ideas to improve add-ons | Asa Dotzler | 3/17/12 5:30 PM | I think making it as you first said it was would be a good step forward.
Can we do that? I'll file the bug. - A |
| Re: A summary of ideas to improve add-ons | Asa Dotzler | 3/17/12 5:58 PM | On 3/14/2012 4:36 PM, Nicholas Nethercote wrote:This is the largest problem we face today with add-ons. I count in "performance" everything from start-up time to GUI and page-load performance to jank caused by excessive memory usage. Any real solution to this problem must either prevent add-ons with these kinds of problems from ever reaching our users or must ensure that our users have real, actionable information about the problems the add-on can cause before installing (for new installations) and after installation (for installations that were in place before the new warning.) Serious problems in this area which aren't covered by performance are stability, Firefox UX breakage, and site breakage. If the add-on's features are not well designed for usability, that is a problem, but if the add-on doesn't cause problems with Firefox and is only a poor experience in and of itself, I think that's less serious. As with the first category, any thorough solution to this problem should prevent add-ons which cause Firefox crashes, or UX or site breakage from reaching users or must ensure that our users have real, actionable information about the problems. If the problems of performance and quality are addressed, either by preventing those problems from reaching users or by informing users about the problems so they can make good choices, this area should be mostly limited to privacy, security, and usability problems. I believe that we have solved some of this problem with prompts to warn users of foreign-installed add-ons. I'm concerned that the implementation there is not as usable/actionable as it could be and I'd like us to invest more in UX/UR to get that as good as it can be. We also can mitigate with easier tools for toggling on and off foreign-installed toolbars. The Australis Firefox re-design includes plans to make this easier. I think we should make this available and we should do pro-active outreach to push this onto non-AMO add-on authors -- at least any with substantial user numbers. This seems like an absolute necessity for providing users with real choice. Listings, review states, and ratings are all necessary. It's the only way we can properly warn them about shortcomings with add-ons. We should surface this information during foreign-install "are you sure" prompts and we should make sure it's easily accessible for add-ons installed from non-AMO websites. For users who already have add-ons installed, we should offer a one-time prompt on a future update that helps them understand the implications of their current add-ons. We should also update the add-on manager to grab relevant data from the listings and display it along side the add-on in the add-ons manager. - A |
| Re: A summary of ideas to improve add-ons | Mark Finkle | 3/17/12 10:07 PM | On 03/15/2012 06:23 AM, Henri Sivonen wrote:I also disagree with this idea. It's one of the reasons add-ons are popular in the first place. |
| Re: A summary of ideas to improve add-ons | Gijs Kruitbosch | 3/18/12 12:04 AM | On 18/03/2012 01:58 AM, Asa Dotzler wrote:
> <snip> > > - provide listings for non-AMO add-ons on AMOHow do we do this? I mean, the add-ons might or might not be signed. To the best of my knowledge, add-on IDs are not secret and anyone can theoretically use any other ID. Couldn't they just use anyone's add-on ID (say, one that's well-rated/reviewed)? Or keep switching IDs? I'm not even entirely sure how we'd go about 'authenticating' an add-on -- even if we force signing them, we'd need to start correlating the signatures with the IDs in some way... (which, to the best of my knowledge, they currently aren't). ~ Gijs |
| Re: A summary of ideas to improve add-ons | Asa Dotzler | 3/18/12 12:38 AM | Like that Google Toolbar that so helpfully replaced your Firefox
bookmarks and history and passwords -- until they discontinued it and you lost everything. Users should understand what they're getting into with add-ons that replace Firefox functionality like that. I don't think most of them do. - A |
| Re: A summary of ideas to improve add-ons | Nicholas Nethercote | 3/18/12 1:20 AM | On Sun, Mar 18, 2012 at 11:58 AM, Asa Dotzler <a...@mozilla.org> wrote:Jorge cited http://blog.mozilla.com/addons/2010/10/06/discontinuing-several-features-of-amo/, which explains how AMO used to do this. The problem is that it perverts the incentives -- add-on authors can get the benefits of being on AMO (visibility) without the costs (having to pass review). This makes it less likely that authors will distribute their add-ons through AMO, which is bad. Maybe the model could be tweaked to avoid this problem -- e.g. allow reviews and a description of an add-on without providing a download link? But that sounds a bit odd. The link also explains that it made AMO significantly more complex. Nick |
| Re: A summary of ideas to improve add-ons | Asa Dotzler | 3/18/12 2:17 AM | On 3/18/2012 1:20 AM, Nicholas Nethercote wrote:
> On Sun, Mar 18, 2012 at 11:58 AM, Asa Dotzler <a...@mozilla.org> wrote: >>> >>> - provide listings for non-AMO add-ons on AMO >> >> This seems like an absolute necessity for providing users with real choice. >> Listings, review states, and ratings are all necessary. It's the only way we >> can properly warn them about shortcomings with add-ons. We should surface >> this information during foreign-install "are you sure" prompts and we should >> make sure it's easily accessible for add-ons installed from non-AMO >> websites. For users who already have add-ons installed, we should offer a >> one-time prompt on a future update that helps them understand the >> implications of their current add-ons. We should also update the add-on >> manager to grab relevant data from the listings and display it along side >> the add-on in the add-ons manager. > > Jorge cited http://blog.mozilla.com/addons/2010/10/06/discontinuing-several-features-of-amo/, > which explains how AMO used to do this. The problem is that it > perverts the incentives -- add-on authors can get the benefits of > being on AMO (visibility) without the costs (having to pass review). > This makes it less likely that authors will distribute their add-ons > through AMO, which is bad. > > Maybe the model could be tweaked to avoid this problem -- e.g. allow > reviews and a description of an add-on without providing a download > link? But that sounds a bit odd. The link also explains that it made > AMO significantly more complex. I mostly care about this data being available so that when a user installs an add-on, we can slurp it into our installation dialog and after we can include it in the add-on manager. We don't have to expose all (or any) of it at AMO. - A |
| Re: A summary of ideas to improve add-ons | Mark Finkle | 3/18/12 7:23 AM | On 03/18/2012 03:38 AM, Asa Dotzler wrote:
> On 3/17/2012 10:07 PM, Mark Finkle wrote: >> On 03/15/2012 06:23 AM, Henri Sivonen wrote: >> >>> add-ons that replace features (like password manager or >>> safe browsing) that Firefox already has built-in. >> >> I also disagree with this idea. It's one of the reasons add-ons are >> popular in the first place. > > Like that Google Toolbar that so helpfully replaced your Firefox > bookmarks and history and passwords -- until they discontinued it and > you lost everything. > > Users should understand what they're getting into with add-ons that > replace Firefox functionality like that. I don't think most of them do. I would argue that point for all add-ons. |
| Re: A summary of ideas to improve add-ons | Dave Townsend | 3/18/12 9:29 AM | On 03/17/12 17:58, Asa Dotzler wrote:
> On 3/14/2012 4:36 PM, Nicholas Nethercote wrote:We have most of the infrastructure in place for this already and we did the one-time prompt with Firefox 8. What new information would you expect us to add to the listings for add-ons on AMO that you think would help users make their choice? |
| Re: A summary of ideas to improve add-ons | Asa Dotzler | 3/18/12 10:25 AM | Performance and quality information for sure. Perhaps warnings about the
problem of putting Firefox core features into the hands of third parties which may not be around for very long. - A |
| Re: A summary of ideas to improve add-ons | Asa Dotzler | 3/18/12 10:26 AM | I don't think it's worth trying to explain to users, for most add-ons,
that if they lose the add-on they'll lose that add-on's functionality. For most add-ons, that's kind of obvious. For add-ons that replace Firefox functionality, though, that loss is entirely different and far more consequential. - A |
| Re: A summary of ideas to improve add-ons | Blair McBride | 3/20/12 3:43 AM | (Apologies for being late to the party - I got swamped, then I got sick;
I'm now catching up between naps.) That model doesn't fit with the problem at hand anyway, since it primarily solves discoverability. Add-ons don't need to be *listed* on AMO for us to get the benefits (reviews, rating, etc). Nor do they need to be submitted. The problem here isn't discoverability/visibility of non-hosted add-ons. All we need is storage of reviews/ratings/etc, and an API. And consumption of that data is contextual: * When you're about to install such an addon * When you already have such an add-on installed And that's *any* non-hosted addon, not just those that were added to AMO's database by the addon author. When we install/detect an addon, we already ask AMO for additional metadata - we can make that include reviews/ratings/etc for non-hosted addons. That ping also gives us a list of non-hosted add-ons to review/test/etc (which has already started). - Blair |
| Re: A summary of ideas to improve add-ons | Nicholas Nethercote | 3/20/12 4:30 AM | On Tue, Mar 20, 2012 at 9:43 PM, Blair McBride <bmcb...@mozilla.com> wrote:That's all true, but if an add-on isn't listed on AMO how/where do users submit reviews and ratings? Nick |
| Re: A summary of ideas to improve add-ons | Jorge Villalobos | 3/20/12 8:04 AM | In the Add-ons Manager?
* ducks * - Jorge |
| Re: A summary of ideas to improve add-ons | Jorge Villalobos | 3/20/12 8:04 AM | On 3/20/12 5:30 AM, Nicholas Nethercote wrote: |
| Re: A summary of ideas to improve add-ons | Dave Townsend | 3/20/12 12:49 PM | The UI can go there sure, we need a service on the web somewhere to
aggregate that data though. AMO already has this functionality. |
| Re: A summary of ideas to improve add-ons | Nicholas Nethercote | 3/20/12 5:53 PM | On Tue, Mar 20, 2012 at 12:49 PM, Dave Townsend <dtow...@mozilla.com> wrote: >> In the Add-ons Manager?Regardless of whether the info is presented on AMO or in the add-ons manager, I imagine users will want to be able to see their reviews and ratings somewhere public and obvious -- it's not much fun to submit reviews/ratings somewhere if those reviews/ratings are only shown to people who install an add-on. (But I could be wrong about this...) But as soon as we make the reviews/ratings public and obvious we're back to the "free visibility for unreviewed add-ons" problem. Nick |
| Re: A summary of ideas to improve add-ons | Nicholas Nethercote | 3/20/12 5:56 PM | On Tue, Mar 20, 2012 at 5:53 PM, Nicholas NethercoteHmm, but maybe if we had "this add-on is great" / "this add-on sucks" buttons in the add-on manager (or maybe a "rate this add-on from 1 to 5 stars" button) that would make rating easy enough that the user wouldn't expect anything once they hit "submit". We could allow a short description to go along with the rating as well. It'd be a bit like the Firefox feedback add-on thingy that is (was?) in beta versions. Nick |
| Re: A summary of ideas to improve add-ons | Blair McBride | 3/21/12 2:50 AM | Yep. I think it only needs to be something lightweight. I don't think
it'd be terribly be useful to let someone see that at 7.13pm last Tuesday they rated Addon Y as "2 stars". It's more useful to show that Addon Y isn't very good and you probably shouldn't install it. - Blair |