Revisiting the idea of rapid releases -> thoughts from a regular user

[shado...@gmail.com]

I've been having a rough few days on the web. Add-ons aren't working right; sciencedirect keeps oddly minimizing things; a captcha which was necessary to get onto a help desk would not work (altho, of course, it works in other browsers), and so on. I tried to install an add-on (Disconnect.Me) just now and it just doesn't seem to work. It's really quite draining.

I haven't really had such odd problems, especially in this variety in this short period of time, in my prior several years of using Firefox, and I'd heard a bit about rapid releases, so I did a search and found the discussion at https://groups.google.com/forum/#!topic/mozilla.dev.planning/19O8ODZnmPo about the extended support release version.

I'm just a regular user. Not an enterprise customer or anything. A wannabe codemonkey but nothing serious. I am afraid that this rapid release system is just too much. I use Firefox for add-ons, mainly, as well as a smooth user experience. If they can't keep up with you guys, then there's no point in using Firefox. And I haven't had a smooth user experience lately. I'm afraid I'm just going to have to give up and use Chrome, which seems more sensible.

Dave Ross and deep64blue consistently pointed out that this rapid release system is as damaging for individuals as for enterprises. Christian basically said: yeah, so what? Short-term pain, long-term gain. No. Well, OK - but when does the "short-term" pain stop? When are you guys going to slow down? And how exactly do you go from releasing after 42 weeks - with memory holes in the release - to releasing after 6 weeks and having no bugs? I just don't get it. Did you hire Superman to code for you?

Christian said that you're not seeing people going back to Firefox 3.6. Well, yeah. That would require a fair bit of effort and knowledge; but even more, users would probably expect to have a less smooth experience with an older browser. So they'll just switch to a different browser.

I was thinking about responding to the thread, but given Christian's very strident commentary that such commentary was off-topic, I'm starting a new thread. I realize there's a decent chance I might get just banned from this group post-haste as I'm not part of your regular group, but I hope not. Slow down, take your time, make sure things work well. There's no big hurry. Updating Firefox quicker is not going to matter to most of your users and it is certainly not going to, say, solve global warming or world hunger. Users want a smooth experience more than the latest bells and whistles. So when you rush things, you're just going to lose out to Chrome.

Companies committing major strategic errors is not uncommon. Clearly, you have a very strong personality (in the form of Christian Legnitto), and from what I saw in the thread dissent is regarded fairly harshly. That's often not a good sign. Also, don't treat your users with contempt. Yes, we're not all codemonkeys. Some of us have time-consuming jobs which do not involve programming. Volunteering, following investments, medical literature, legal updates, etc and so on take up a lot of people outside the code world's time, and you should try to respect that. If you guys have programmers who are antsy and need something to do, I can point you to various projects. Wikipedia desperately needs to be upgraded to Web 2.0, for example, and guess what - that could actually solve some of the world's problems.

Anyway, I know I'm being repetitive. It's been a long day. Thanks.

[Christian Legnitto]

As I have been called out multiple times, I'll comment in line.

On Jan 12, 2012, at 8:05 PM, shado...@gmail.com wrote:

> I've been having a rough few days on the web.

Sorry to hear that!

> Add-ons aren't working right;

Can you be more specific?

> sciencedirect keeps oddly minimizing things

The website? Or an add-on? The PDFs that are downloaded from the site? You might want to see if other people are seeing the issue @ https://support.mozilla.org.

> a captcha which was necessary to get onto a help desk would not work (altho, of course, it works in other browsers)

Sorry, I know this sucks. Were there any errors in the web console (Tools > Web Developer > Error Console) ?

> , and so on. I tried to install an add-on (Disconnect.Me) just now and it just doesn't seem to work

I tried it too. It has been marked compatible with the latest release and installs on Firefox (9.0.1, 10, etc). It looks like it just flat out doesn't work...not because of Firefox (by the looks of it) but just the extension itself. I got the button (which you can barely see) to give me a pop-up once but it looks useless (everything @ 0 blocked).

I think your implication that this was caused by the new process would be much stronger if had worked up until a recent Firefox update. With the information you have given there is no way to tell if the same issue would happen in the old process.

There are a bunch of other add-ons that do what Disconnect.Me do, most likely better (if popularity is any indication). Also, please be sure to turn on http://support.mozilla.org/en-US/kb/how-do-i-turn-do-not-track-feature if you want to signal to advertisers not to track you.

> It's really quite draining.

We've all had those days.

> I haven't really had such odd problems, especially in this variety in this short period of time, in my prior several years of using Firefox, and I'd heard a bit about rapid releases, so I did a search and found the discussion at https://groups.google.com/forum/#!topic/mozilla.dev.planning/19O8ODZnmPo about the extended support release version.

You took a step that many users do not, to understand the "why". Thank you, much appreciated.

> I'm just a regular user. Not an enterprise customer or anything. A wannabe codemonkey but nothing serious. I am afraid that this rapid release system is just too much. I use Firefox for add-ons, mainly, as well as a smooth user experience. If they can't keep up with you guys, then there's no point in using Firefox.

a) The example you have given doesn't support your conclusion. That is, Disconnect.Me wasn't working perfectly fine before and then stopped after an update (unless I misread something). I don't see how you can extrapolate from one (potentially unrelated) add-on experience to a systemic issue

b) For add-ons that do legitimately fall behind, we have been working to help.

First, we created an automatic compatibility checker/changer that is run against all add-ons hosted on addons.mozilla.org (http://groups.google.com/group/mozilla.dev.planning/browse_thread/thread/dafd0bc59b587e27). The vast majority of add-ons on AMO are marked compatible without the developer having to even lift a finger.

Then we created the "Default to Compatible" Firefox feature (https://wiki.mozilla.org/Features/Add-ons/Add-ons_Default_to_Compatible). It is currently in Firefox Beta getting tested for Firefox 10. If you are experiencing issues with incompatible add-ons (or add-ons that are marked compatible but don't work) please let us know! You can help us test Firefox Beta by installing https://mozilla.org/beta (your settings will be saved and you can go back to using the latest supported release by downloading it from mozilla.org).

> And I haven't had a smooth user experience lately. I'm afraid I'm just going to have to give up and use Chrome, which seems more sensible.

That's the beauty of user choice and web standards...try a bunch and choose which suits you best! If Firefox was working for you previously I think it is still likely the best choice, but it doesn't hurt to see what else is out there. Plus, if you installed Adobe Flash via Adobe's site you likely already have Chrome installed due to their shady bundling practices. So easy!

> Dave Ross and deep64blue consistently pointed out that this rapid release system is as damaging for individuals as for enterprises.

Not with compelling arguments FWIW.

> Christian basically said: yeah, so what? Short-term pain, long-term gain.

No, I didn't say "so what". I said we are aware of the trade-offs and the direction was a conscious choice. If there was a clear easy/right/painless way...we'd have done it.

> No. Well, OK - but when does the "short-term" pain stop?

We're just about through it. Add-ons have been keeping up with the release process (and 3rd party developers are changing their ways for the better). Users and web developers are learning to ignore version numbers. We have the "Default to Compatible" feature (again, https://wiki.mozilla.org/Features/Add-ons/Add-ons_Default_to_Compatible) in Firefox Beta for testing. We have a large chunk of the "silent updates" feature (http://blog.mozilla.com/futurereleases/2011/10/04/mozilla-firefox-and-silent-updates/) on our development branch.

> When are you guys going to slow down?

We're not. Well, not without good reasons and data in any case. We have only started to have the data to determine what works best and where the sweet spot is. Unless there is compelling evidence, we're sticking with the current plan (plus the ESR as previous mentioned to help enterprises as they have unique requirements).

> And how exactly do you go from releasing after 42 weeks - with memory holes in the release - to releasing after 6 weeks and having no bugs?

a) We could *not* continue to ship every 42 weeks and be relevant. Period. Mitchell explains more in this blog post: http://blog.lizardwrangler.com/2011/08/25/rapid-release-process/

b) No software wil ever have zero bugs, especially a web browser that runs on multiple platforms (which have bugs) and allows plugins (which have bugs) and add-ons (which have bugs) implementing ever changing web standards (which sometimes have bugs in the specs!)

c) If you think Firefox had "memory holes" in Firefox 4 (the last release in the old process) you should be *ecstatic* about the new process. If we were on Firefox 4.0.11 (as we would in the old process) the claimed "memory holes" would most likely still be there. If you look back at the release notes for Firefox 5+, you'll see drastic reductions in memory use (as an example, here's details about Firefox 7: http://hacks.mozilla.org/2011/09/firefox-7-is-lean-and-fast/).

Don't take my word for it:

https://www.google.com/search?q=firefox+memory+improvements

Of course, there is always more to do...

> I just don't get it.

It's ok if you don't get it, finding this newsgroup and posting will help you (and others searching!) get what we get.

> Did you hire Superman to code for you?

Superman would probably have shit code. Plus, even with all his strength I doubt he could take on the behemoth that is Bugzilla!

> Christian said that you're not seeing people going back to Firefox 3.6.

Yep, still true.

> Well, yeah. That would require a fair bit of effort and knowledge;

Or searching? https://www.google.com/search?q=firefox+3.6 gives me the correct link (http://www.mozilla.org/en-US/firefox/all-older.html) as the first hit.

We're actually still supporting 3.6 while we implement ESR to completely remove enterprise worries about a quick transition.

> but even more, users would probably expect to have a less smooth experience with an older browser. So they'll just switch to a different browser.

Yep, which is exactly what would happen if we had a Firefox 4.0.11 or a Firefox 5.0.3 in the market today. Our users would constantly be 3-12 months behind on our engineering work and as a result would suffer and look elsewhere. Plus, we wouldn't compare favorably when comparing browsers.

For example, if a 3.6 (or 4.0.11 or 5) user tries the latest [insert any browser] they will be *wowed* by how fast it is. More of their HTML5 sites will now "just work" and work better. If they didn't know about Firefox 9, they might be a new [insert any browser] user for life ("I remember Firefox, it was so slow!"). But if a Firefox 9 user checks out the latest [insert any browser], the performance and feature difference is much more of a wash (heavily dependent on workload) and we have a better chance of retaining that loyal user.

We also don't want our awesome Firefox users who don't want to switch (because of add-ons, compatibility, Sync, etc) to have a *worse* performance and web experiences...which is what would happen with slower releases.

For a concrete example, consider this:

Without faster releases Google Maps GL (http://googlesystem.blogspot.com/2011/10/google-maps-in-webgl.html) wouldn't work in Firefox as support for underlying technology (CORS in WebGL) was only recently standardized and then implemented by browsers. Why should a user of the latest Chrome get to use 3D map goodness while users of the latest Firefox need to wait because of something as silly as a release process? With the new process as soon as we finished the engineering and testing our users saw the benefit rather than us keeping it locked up for 42 weeks in our development channel.

> I was thinking about responding to the thread, but given Christian's very strident commentary that such commentary was off-topic, I'm starting a new thread.

That was an ESR thread. I reminded people that we were specifically *not* using ESR as a way to change the new release process. It is an added / separate initiative targeted to a specific use case we know isn't covered by the faster process. I wanted the discussion to focus on that specific need and if the proposal was sufficient to address it.

> I realize there's a decent chance I might get just banned from this group post-haste as I'm not part of your regular group, but I hope not.

We encourage discussion, especially if it is calm and collected like yours. Open discussion and participation is one of the greatest strengths of Mozilla. That is, unless you use the term "version inflation"...that's an instant ban as far as I am concerned (kidding! ...kinda).

> Slow down, take your time, make sure things work well. There's no big hurry. Updating Firefox quicker is not going to matter to most of your users and it is certainly not going to, say, solve global warming or world hunger. Users want a smooth experience more than the latest bells and whistles. So when you rush things, you're just going to lose out to Chrome.

We're not rushing, we're merely choosing to cut the development into 7 smaller 6 week releases instead of one massive 42 week release. The engineering work is roughly the same and the new process arguably has more checking / escape valves built in (on purpose!).

> Companies committing major strategic errors is not uncommon.

Yep. And I would argue doing nothing due to paralysis or maintaining the status quo has been the cause of more downfalls than the opposite. Mozilla as a project has a tendency to lean towards no change (see http://omgchange.com/, set up before the new release process by other Mozillians). Go big or go home as they say.

That being said, we are watching data very, very closely.

> Clearly, you have a very strong personality (in the form of Christian Legnitto)

I hope it came off as knowledgable yet firm rather than rude. If not, I apologize.

> and from what I saw in the thread dissent is regarded fairly harshly.

No. Rehashing the same arguments that have already been discussed, acknowledged, and dismissed (for various reasons) in a thread with a very specific purpose is treated harshly. We're herding cats here, sometimes we need to get the discussion back on the rails (also see http://en.wiktionary.org/wiki/bikeshedding, a term I wasn't aware of until coming here).

> That's often not a good sign.

Yes, silencing debate and critics is not. Telling people to read the older discussions or bring up new data for a issue that is already decided is the way you actually get stuff done.

> Also, don't treat your users with contempt.

I can assure you, we do not. We're not some big mega-corp. We're an open source project trying to do what's best for the web. Our friends, family, and relatives use Firefox and we want it to be great for them.

> Anyway, I know I'm being repetitive. It's been a long day. Thanks.

We've all had those. Hope your evening is better and I truly hope this response sheds some light on previous discussions.

Thanks,
Christian Legnitto
Firefox Release Manager

[Nicholas Nethercote]

On Thu, Jan 12, 2012 at 10:14 PM, Christian Legnitto
<cleg...@mozilla.com> wrote:
>
>> And how exactly do you go from releasing after 42 weeks - with memory holes in the release - to releasing after 6 weeks and having no bugs?
>

> c) If you think Firefox had "memory holes" in Firefox 4 (the last release in the old process) you should be *ecstatic* about the new process. If we were on Firefox 4.0.11 (as we would in the old process) the claimed "memory holes" would most likely still be there. If you look back at the release notes for Firefox 5+, you'll see drastic reductions in memory use (as an example, here's details about Firefox 7: http://hacks.mozilla.org/2011/09/firefox-7-is-lean-and-fast/).

Exactly. Firefox 4 was awful from a memory POV. Firefox 5 was a
little better. Firefox 7 was way better. Firefox 8 and 9 and 10 are
each better again.
Rapid release is your friend here.

Furthermore, the reason Firefox 4 was so awful was that a ton of new
features went in all at once that hurt memory consumption, and this
wasn't realized until relatively late in the cycle because we didn't
have many beta testers. With the rapid releases, only a small number
of new features go into each release and then that release is used by
a wide audience, so we get feedback (including telemetry data) in
small, regular doses, which makes it much easier to measure progress
and catch regressions.

Nick