> we've discussed before if we're doing the update from 3.5 to 3.6 as > minor update.
Indeed, and this came up again in today's development meeting. I should have replied to this thread sooner.
The motivation remains the same as when the notion was first discussed early in the Firefox 3.6 development cycle: to quickly migrate our existing user base to Firefox 3.6 / mozilla-1.9.2 based releases.
To date we have offered our users two different types of updates, which we call "major" and "minor". Major updates have been for code which has been in development for at least 12 months, features visible changes to the user interface either in terms or appearance or interaction, and/or contains major technology changes at the platform level which may cause significant differences in system requirements or support requirements. Minor updates have been for security and stability releases which do not contain any visible user interface changes, and are limited in terms of technological change.
We offer major updates as a choice that the user must accept, due to our belief that the user should be able to decide whether or not they want the new features and the change to their interactions. We force minor updates without a choice (though all application updating can be disabled) because we believe that the updates are important for users, and that the new version of the product comes with all the benefits of the older version without any additional cost. Further, we force minor updates because we know that simply by virtue of offering a choice, fewer users will accept the new version, and many of the users who reject it will not reject it for the correct reasons (because they don't understand the dialog, are resistant to change, do not understand the costs and benefits, etc.)
The pace of technology development in web browsers is speeding up rapidly, and we now face a challenge of ensuring that we can continue to deliver modern web browsing experiences to our users. Many of these new experiences do not result in any major user interface changes, instead relying on improving performance, stability and enabling new web technology features. As proposed earlier in the summer, Firefox 3.6 will be primarily a release with security, stability, speed and capability enhancements, with no visible user interface changes over Firefox 3.5. As such, I think we should consider it as a candidate for a minor update, stretching our definition of what types of updates we can provide using that mechanism.
Finally, users' expectations of software have changed since the update mechanism was introduced in Firefox 1.5. Many applications that browser users interact with exist in the cloud, with updates pushed frequently and transparently, without consultation. That wasn't the case only a few years ago.
> Do we know what's going to drive that decision? I know that java was in > the game, that seems to be solved.
The factors that will drive that decision are:
- compatibility concerns - stability and user testing concerns - logistics and timing concerns
// Compatibility concerns
Add-on compatibility is one of the large reasons why users do not move from one version to another. Some (but few) minor updates break Add-on compatibility, but to date we've had trouble ensuring that major updates do not leave Add-ons in an incompatible state. As such, we've been overly conservative, waiting for Add-on authors to mark their software as compatible with the new version; this can take a long time. To help with this, the Add-ons team will be creating a crowdsourcing application that will allow beta testers to let us know if Add-ons work as expected, thus reducing the testing requirements on Add-on authors themselves.
Website compatibility is another issue, though our QA team has solved this by collecting data on commonly used websites and ensuring that our new products remain compatible with those sites before we release. We also depend on feedback during the beta cycle.
Finally, throughout the development cycle of Firefox 3.6 we have been taking fewer invasive changes which should make it easier to reason about and test for compatibility concerns.
// Stability and user testing concerns
Firefox 3.6 will have a shorter beta cycle than other releases, which is a concern due to the number of blocking issues that have been found in previous betas. However, as stated above, since we have taken fewer invasive changes, it should be easier to focus our beta audience and the areas for which we are concerned.
A recent focus on crash monitoring and analysis should give us better tools to reason about stability of the product, but fundamentally, I have problems with the idea that we'd consider a product "stable enough" to put up on our website for download, yet not have the confidence to issue it as an update to our user base.
// Logistics and timing concerns
We still need to do quite a bit of testing to ensure that a minor update of this scale will work as expected. Further, if we were to ship it as a minor update we would need to do so at a time when there will be enough staff on hand to deal with questions and issues that may arise; depending on the ship schedule for Firefox 3.6, this may force the update to be some time after the product is available for download.
> If we're doing it, by when, and by which criteria?
This also hasn't been fully fleshed out, but it's possible to release Firefox 3.6 and not update Firefox 3.5 users immediately. In the past we've avoided doing that for bandwidth preservation reasons alone, and we may wish to continue with that policy.
> Firefox 3.6 will have a shorter beta cycle than other releases, which is > a concern due to the number of blocking issues that have been found in > previous betas.
Based on what I've seen so far, 3.6b1 resembles previous 3.Xb1 in that it will be (from my perspective) not significantly better than nightly builds. It seems that a lot of time and energy goes into the b1, but it falls short. I don't think this is just "duh". Rather I think the nightly build infrastructure is really excellent as far as it goes, but whatever is going on after that does not add enough value at the b1 stage. The b1 becomes an instant reject because some few aspects don't work for extensions which are vital to the beta user community. In future, consider putting less energy into b1 and more energy into shortening the distance between b1 and b2 where extensions have had a chance to get fixes in. Perhaps even invent a "bx" release, explicitly for this purpose.
In today's development meeting, there was some discussions about Scenario#1 and Scenario#2 below. I'd like to propose Scenario#3, which afaict meets the concerns raised.
Scenario#1: =========== - on day of release of FF3.6.0, existing FF3.5.x users get a *minor* update to FF3.6.0. ...which means: -- by using that minor upgrade dialog, user will find it harder to see what changed until *after* installation -- users could be surprised by change in functionality (and UI?) when they are used to only get small security fixes in a minor update scenario. Downgrading is tricker? Would this surprise encourage users to turn off getting Firefox updates in future? -- concerns about new problems discovered only after a lot of users were broken. Bad user experience. -- mozilla will need to support one less branch. We still support FF3.0.x, FF3.6.x but would no longer need to support FF3.5.x.
Scenario#2: =========== - on day of release of FF3.6.0, existing FF3.5.x users get a prompted *major* update. ...which means: -- upgrade dialog box has details of what will change. -- users would not be surprised by change. -- mozilla will need to support FF3.5 in addition to FF3.0.x, FF3.6.x. -- concerns about new problems discovered only after a lot of users were broken. Bad user experience. -- maybe less risk if we did this with FF3.6.1 instead?
Scenario#3: =========== - on day of release of FF3.6.0, existing FF3.5.x users are not prompted to update, but users can choose to manually-only *major* update to FF3.6.0. This is just like we did for the release of FF3.5.0, was quite used for 35-45% of upgrades, and we know our infrastructure can do this safely. - early adopters would pro-actively upgrade to 3.6.0 - we monitor for crash reports or any problems. -- if there are no red flags after a week, we can proceed to: -- start a *gradual* rollout of idle-background *major* update. AUS allows us to gradually un-throttle MU prompts to idle-users, so only a subset of users would see the prompt. Maybe start with 10%? - if there are no red flags after another week, we can raise to 20%, then 50%, then 100%. ...which means: -- we have time to react to surprise bugs if needed -- we can reverse throttling if needed to limit exposure to surprise bugs as they are discovered. -- user-visible upgrade dialog box has details of what changed. -- we offer MU more quickly then we did for FF3.0->FF3.5 -- mozilla will need to support FF3.5 in addition to FF3.0.x, FF3.6.x.
Scenario#3 seems to meet the concerns I've heard in recent weeks, and again today, but let me know if I missed/misunderstood anything.
Mike Beltzner wrote: > On 10/15/2009 8:05 PM, Axel Hecht wrote: >> we've discussed before if we're doing the update from 3.5 to 3.6 as >> minor update.
> Indeed, and this came up again in today's development meeting. I should > have replied to this thread sooner.
> The motivation remains the same as when the notion was first discussed > early in the Firefox 3.6 development cycle: to quickly migrate our > existing user base to Firefox 3.6 / mozilla-1.9.2 based releases.
> To date we have offered our users two different types of updates, which > we call "major" and "minor". Major updates have been for code which has > been in development for at least 12 months, features visible changes to > the user interface either in terms or appearance or interaction, and/or > contains major technology changes at the platform level which may cause > significant differences in system requirements or support requirements. > Minor updates have been for security and stability releases which do not > contain any visible user interface changes, and are limited in terms of > technological change.
> We offer major updates as a choice that the user must accept, due to our > belief that the user should be able to decide whether or not they want > the new features and the change to their interactions. We force minor > updates without a choice (though all application updating can be > disabled) because we believe that the updates are important for users, > and that the new version of the product comes with all the benefits of > the older version without any additional cost. Further, we force minor > updates because we know that simply by virtue of offering a choice, > fewer users will accept the new version, and many of the users who > reject it will not reject it for the correct reasons (because they don't > understand the dialog, are resistant to change, do not understand the > costs and benefits, etc.)
> The pace of technology development in web browsers is speeding up > rapidly, and we now face a challenge of ensuring that we can continue to > deliver modern web browsing experiences to our users. Many of these new > experiences do not result in any major user interface changes, instead > relying on improving performance, stability and enabling new web > technology features. As proposed earlier in the summer, Firefox 3.6 will > be primarily a release with security, stability, speed and capability > enhancements, with no visible user interface changes over Firefox 3.5. > As such, I think we should consider it as a candidate for a minor > update, stretching our definition of what types of updates we can > provide using that mechanism.
> Finally, users' expectations of software have changed since the update > mechanism was introduced in Firefox 1.5. Many applications that browser > users interact with exist in the cloud, with updates pushed frequently > and transparently, without consultation. That wasn't the case only a few > years ago.
>> Do we know what's going to drive that decision? I know that java was in >> the game, that seems to be solved.
> The factors that will drive that decision are:
> - compatibility concerns > - stability and user testing concerns > - logistics and timing concerns
> // Compatibility concerns
> Add-on compatibility is one of the large reasons why users do not move > from one version to another. Some (but few) minor updates break Add-on > compatibility, but to date we've had trouble ensuring that major updates > do not leave Add-ons in an incompatible state. As such, we've been > overly conservative, waiting for Add-on authors to mark their software > as compatible with the new version; this can take a long time. To help > with this, the Add-ons team will be creating a crowdsourcing application > that will allow beta testers to let us know if Add-ons work as expected, > thus reducing the testing requirements on Add-on authors themselves.
> Website compatibility is another issue, though our QA team has solved > this by collecting data on commonly used websites and ensuring that our > new products remain compatible with those sites before we release. We > also depend on feedback during the beta cycle.
> Finally, throughout the development cycle of Firefox 3.6 we have been > taking fewer invasive changes which should make it easier to reason > about and test for compatibility concerns.
> // Stability and user testing concerns
> Firefox 3.6 will have a shorter beta cycle than other releases, which is > a concern due to the number of blocking issues that have been found in > previous betas. However, as stated above, since we have taken fewer > invasive changes, it should be easier to focus our beta audience and the > areas for which we are concerned.
> A recent focus on crash monitoring and analysis should give us better > tools to reason about stability of the product, but fundamentally, I > have problems with the idea that we'd consider a product "stable enough" > to put up on our website for download, yet not have the confidence to > issue it as an update to our user base.
> // Logistics and timing concerns
> We still need to do quite a bit of testing to ensure that a minor update > of this scale will work as expected. Further, if we were to ship it as a > minor update we would need to do so at a time when there will be enough > staff on hand to deal with questions and issues that may arise; > depending on the ship schedule for Firefox 3.6, this may force the > update to be some time after the product is available for download.
>> If we're doing it, by when, and by which criteria?
> This also hasn't been fully fleshed out, but it's possible to release > Firefox 3.6 and not update Firefox 3.5 users immediately. In the past > we've avoided doing that for bandwidth preservation reasons alone, and > we may wish to continue with that policy.
> we've discussed before if we're doing the update from 3.5 to 3.6 as > minor update. On Wed, Oct 21, 2009 at 7:12 AM, Mike Beltzner <beltz...@mozilla.com> wrote: > Indeed, and this came up again in today's development meeting. I should have > replied to this thread sooner. > The factors that will drive that decision are: > - compatibility concerns > - stability and user testing concerns > - logistics and timing concerns
> // Compatibility concerns > Add-on compatibility is one of the large reasons why users do not move from > one version to another. Some (but few) minor updates break Add-on > compatibility, but to date we've had trouble ensuring that major updates do > not leave Add-ons in an incompatible state. As such, we've been overly > conservative, waiting for Add-on authors to mark their software as > compatible with the new version; this can take a long time. To help with > this, the Add-ons team will be creating a crowdsourcing application that > will allow beta testers to let us know if Add-ons work as expected, thus > reducing the testing requirements on Add-on authors themselves.
It might be worth noting that at Nokia, we hit a huge rough patch trying to update from 1.9.2pre to 1.9.2, namely, another group had a NPAPI plugin using the XPCOM Scriptable Plugin solution (obsolete pre Firefox 1.0, but they didn't ask us, and really refuse to let us manage them or their priorities). For anyone using this plugin, the update isn't a minor update, since it breaks it entirely (this has effectively blocked our ability to upgrade to a remotely safe version of Gecko for MicroB on the N900).
That group FWIW, seems to be interested in redoing their plugin to work with a Nokia initiative (CWRT) which seems to mean that they don't intend to deliver anything useful for a long time.
From a certain perspective, I'd actually encourage you to upgrade everyone to FF3.6 so that these people will understand that their priorities are backwards. OTOH, if you actually care about people who might be using this plugin, you might want to do something else.
Oh, and as for confidentiality, this isn't. >300 people not working for Nokia have N900s with Nokia Maps, and anyone can take a debug pluginhost to the maps plugin and determine that it uses xpcom scripting (or they could just look for the .xpt file!). As for CWRT, "nokia common wrt", http://justamp.blogspot.com/2009/01/nokia-wrt-plug-in-for-aptana-stud... it's also public knowledge.
I'd like to re-raise another concern, and that's product availability and quality per localization.
I don't have reliable numbers on that right now, and given our timeframe, I don't think I'll have some until really close to RC1. I know at least one team where we won't have strong ownership 'til then. What I know is that we have less than 40 locales with nightly testers, and it's more like 10 locales with more than a dozen in the last week.
I would add that I recall our talks about this to be more towards doing that upgrade on 3.6.1 than 3.6.0, and that framed the amount of outreach we did so far, too.
We'll surely try to be up-to-par on 3.6 compared to 3.5 as quickly as we can, but I don't have an ETA or a "that's how many users would get X experience" until a good deal further down the road.
> On 10/15/2009 8:05 PM, Axel Hecht wrote: >> we've discussed before if we're doing the update from 3.5 to 3.6 as >> minor update.
> Indeed, and this came up again in today's development meeting. I should > have replied to this thread sooner.
> The motivation remains the same as when the notion was first discussed > early in the Firefox 3.6 development cycle: to quickly migrate our > existing user base to Firefox 3.6 / mozilla-1.9.2 based releases.
> To date we have offered our users two different types of updates, which > we call "major" and "minor". Major updates have been for code which has > been in development for at least 12 months, features visible changes to > the user interface either in terms or appearance or interaction, and/or > contains major technology changes at the platform level which may cause > significant differences in system requirements or support requirements. > Minor updates have been for security and stability releases which do not > contain any visible user interface changes, and are limited in terms of > technological change.
> We offer major updates as a choice that the user must accept, due to our > belief that the user should be able to decide whether or not they want > the new features and the change to their interactions. We force minor > updates without a choice (though all application updating can be > disabled) because we believe that the updates are important for users, > and that the new version of the product comes with all the benefits of > the older version without any additional cost. Further, we force minor > updates because we know that simply by virtue of offering a choice, > fewer users will accept the new version, and many of the users who > reject it will not reject it for the correct reasons (because they don't > understand the dialog, are resistant to change, do not understand the > costs and benefits, etc.)
> The pace of technology development in web browsers is speeding up > rapidly, and we now face a challenge of ensuring that we can continue to > deliver modern web browsing experiences to our users. Many of these new > experiences do not result in any major user interface changes, instead > relying on improving performance, stability and enabling new web > technology features. As proposed earlier in the summer, Firefox 3.6 will > be primarily a release with security, stability, speed and capability > enhancements, with no visible user interface changes over Firefox 3.5. > As such, I think we should consider it as a candidate for a minor > update, stretching our definition of what types of updates we can > provide using that mechanism.
> Finally, users' expectations of software have changed since the update > mechanism was introduced in Firefox 1.5. Many applications that browser > users interact with exist in the cloud, with updates pushed frequently > and transparently, without consultation. That wasn't the case only a few > years ago.
>> Do we know what's going to drive that decision? I know that java was in >> the game, that seems to be solved.
> The factors that will drive that decision are:
> - compatibility concerns > - stability and user testing concerns > - logistics and timing concerns
> // Compatibility concerns
> Add-on compatibility is one of the large reasons why users do not move > from one version to another. Some (but few) minor updates break Add-on > compatibility, but to date we've had trouble ensuring that major updates > do not leave Add-ons in an incompatible state. As such, we've been > overly conservative, waiting for Add-on authors to mark their software > as compatible with the new version; this can take a long time. To help > with this, the Add-ons team will be creating a crowdsourcing application > that will allow beta testers to let us know if Add-ons work as expected, > thus reducing the testing requirements on Add-on authors themselves.
> Website compatibility is another issue, though our QA team has solved > this by collecting data on commonly used websites and ensuring that our > new products remain compatible with those sites before we release. We > also depend on feedback during the beta cycle.
> Finally, throughout the development cycle of Firefox 3.6 we have been > taking fewer invasive changes which should make it easier to reason > about and test for compatibility concerns.
> // Stability and user testing concerns
> Firefox 3.6 will have a shorter beta cycle than other releases, which is > a concern due to the number of blocking issues that have been found in > previous betas. However, as stated above, since we have taken fewer > invasive changes, it should be easier to focus our beta audience and the > areas for which we are concerned.
> A recent focus on crash monitoring and analysis should give us better > tools to reason about stability of the product, but fundamentally, I > have problems with the idea that we'd consider a product "stable enough" > to put up on our website for download, yet not have the confidence to > issue it as an update to our user base.
> // Logistics and timing concerns
> We still need to do quite a bit of testing to ensure that a minor update > of this scale will work as expected. Further, if we were to ship it as a > minor update we would need to do so at a time when there will be enough > staff on hand to deal with questions and issues that may arise; > depending on the ship schedule for Firefox 3.6, this may force the > update to be some time after the product is available for download.
>> If we're doing it, by when, and by which criteria?
> This also hasn't been fully fleshed out, but it's possible to release > Firefox 3.6 and not update Firefox 3.5 users immediately. In the past > we've avoided doing that for bandwidth preservation reasons alone, and > we may wish to continue with that policy.
Mike Beltzner wrote: > On 10/15/2009 8:05 PM, Axel Hecht wrote: >> we've discussed before if we're doing the update from 3.5 to 3.6 as >> minor update.
> Indeed, and this came up again in today's development meeting. I > should have replied to this thread sooner.
> The motivation remains the same as when the notion was first discussed > early in the Firefox 3.6 development cycle: to quickly migrate our > existing user base to Firefox 3.6 / mozilla-1.9.2 based releases.
> To date we have offered our users two different types of updates, > which we call "major" and "minor". Major updates have been for code > which has been in development for at least 12 months, features visible > changes to the user interface either in terms or appearance or > interaction, and/or contains major technology changes at the platform > level which may cause significant differences in system requirements > or support requirements. Minor updates have been for security and > stability releases which do not contain any visible user interface > changes, and are limited in terms of technological change.
> We offer major updates as a choice that the user must accept, due to > our belief that the user should be able to decide whether or not they > want the new features and the change to their interactions. We force > minor updates without a choice (though all application updating can be > disabled) because we believe that the updates are important for users, > and that the new version of the product comes with all the benefits of > the older version without any additional cost. Further, we force minor > updates because we know that simply by virtue of offering a choice, > fewer users will accept the new version, and many of the users who > reject it will not reject it for the correct reasons (because they > don't understand the dialog, are resistant to change, do not > understand the costs and benefits, etc.)
> The pace of technology development in web browsers is speeding up > rapidly, and we now face a challenge of ensuring that we can continue > to deliver modern web browsing experiences to our users. Many of these > new experiences do not result in any major user interface changes, > instead relying on improving performance, stability and enabling new > web technology features. As proposed earlier in the summer, Firefox > 3.6 will be primarily a release with security, stability, speed and > capability enhancements, with no visible user interface changes over > Firefox 3.5. As such, I think we should consider it as a candidate for > a minor update, stretching our definition of what types of updates we > can provide using that mechanism.
> Finally, users' expectations of software have changed since the update > mechanism was introduced in Firefox 1.5. Many applications that > browser users interact with exist in the cloud, with updates pushed > frequently and transparently, without consultation. That wasn't the > case only a few years ago.
>> Do we know what's going to drive that decision? I know that java was in >> the game, that seems to be solved.
> The factors that will drive that decision are:
> - compatibility concerns > - stability and user testing concerns > - logistics and timing concerns
> // Compatibility concerns
> Add-on compatibility is one of the large reasons why users do not move > from one version to another. Some (but few) minor updates break Add-on > compatibility, but to date we've had trouble ensuring that major > updates do not leave Add-ons in an incompatible state. As such, we've > been overly conservative, waiting for Add-on authors to mark their > software as compatible with the new version; this can take a long > time. To help with this, the Add-ons team will be creating a > crowdsourcing application that will allow beta testers to let us know > if Add-ons work as expected, thus reducing the testing requirements on > Add-on authors themselves.
> Website compatibility is another issue, though our QA team has solved > this by collecting data on commonly used websites and ensuring that > our new products remain compatible with those sites before we release. > We also depend on feedback during the beta cycle.
> Finally, throughout the development cycle of Firefox 3.6 we have been > taking fewer invasive changes which should make it easier to reason > about and test for compatibility concerns.
> // Stability and user testing concerns
> Firefox 3.6 will have a shorter beta cycle than other releases, which > is a concern due to the number of blocking issues that have been found > in previous betas. However, as stated above, since we have taken fewer > invasive changes, it should be easier to focus our beta audience and > the areas for which we are concerned.
> A recent focus on crash monitoring and analysis should give us better > tools to reason about stability of the product, but fundamentally, I > have problems with the idea that we'd consider a product "stable > enough" to put up on our website for download, yet not have the > confidence to issue it as an update to our user base.
All the above is really good analysis. On the last point I'll agree that the tools are getting better but the facts remain that we need to put all the crash fixes, compatibility fixes, security fixes out to a wide audiances, then provide some time to get those things in use, then assess. The tools we have can't preemptively measure how good we are on each of these stability and performance vectors for the whole body of the web and I think there is a long road to get them to that state. Our efforts to change from a addon compatibility model where the author confirms compatibility, to a model of assumed compatibility unless users tells about the bad ones now puts addons in the same boat as the stability and performance assessments. We'll need a body of 3, to 5, to 10 milllion users to tell us how we are doing there to understand and assess where we are.
I also agree that all the browser makers want to, and are moving much faster.
But I'll submit that browsers users might be organizing into three camps, and I'll steal from Andrea Balle's analysis to describe them
There is the IE6 camp. These are the 'old volkswagon' users of the web. The have some kind of strange comfort in something old and familiar and available. There is not much that we should be doing for these users other than to convince them to move to a modern browser.
There is the 'high performance sedan users of the web. This set of users wants to drive the 5 passenger high performance sedan on the autoban of the web in extreme comfort. They get extremely annoyed by shakes, rattles, stability, compatibility and performance problems. Everything should work flawlessly and be well designed and integrated.
There are the 'browser racing enthusiasts' of the web. These are the forumal 1 browser users. The enjoy participating in the advancements, thrills, and chills of the browser and web development. If there is not a high pace of speed and advancement, and the occasional spectacular blowout wreck, things just aren't moving fast enough. Here is one of the many voices from the race track... http://blog.internetnews.com/skerner/2009/10/mozilla-firefox-354-beta...
I think we might be trying to make something in the middle. Its not moving fast enough for the browser enthusiasts, and the fewer beta's, and shorter beta bake times, along with and quick ramp up in update migrations have the potencial to expose the 'high performance sedan' users annoyances they just don't want to see.
Like the auto industry, I wonder if we to figure out how we could create test vehicles for the track that had just the following of the browser enthusiasts. A path like that could allow us to make improvements that have the longer bake time needed to serve the 'high performance sedan' browser users better.
This is all just theory and the classifications might not really exist. One way to test this would be try and figure out these kind of users acutally exist, and to find out how many of them there are out on the web.
Market share analysis might tell us that 30% are still old volkswagon users. Googles 'what is a browser?' poll suggests that a only 90% of users might not even know what a browser is. To me that suggests a pretty high percentage might be in high performance sedan browser classification. I also bet there are maybe 10 to 15 millon browser enthusiasts out there. Firefox still holds a good many of these, and I think we need to figure out how to do a better job of organizing them and harnessing them to help us understand exactly where we are in terms of stability, performance, security, and the other vectors that people assess browser releases on before we unleash the a browser update to the high perfomance sedan users.
> We still need to do quite a bit of testing to ensure that a minor > update of this scale will work as expected. Further, if we were to > ship it as a minor update we would need to do so at a time when there > will be enough staff on hand to deal with questions and issues that > may arise; depending on the ship schedule for Firefox 3.6, this may > force the update to be some time after the product is available for > download.
>> If we're doing it, by when, and by which criteria?
> This also hasn't been fully fleshed out, but it's possible to release > Firefox 3.6 and not update Firefox 3.5 users immediately. In the past > we've avoided doing that for bandwidth preservation reasons alone, and > we may wish to continue with that policy.
Mike Beltzner wrote: > Finally, throughout the development cycle of Firefox 3.6 we have been > taking fewer invasive changes which should make it easier to reason > about and test for compatibility concerns.
But I think enough that it upset quite a number of people if we silently force those changes upon them.
John O'Duinn wrote: > Scenario#1: > =========== > [...] > -- mozilla will need to support one less branch. We still support > FF3.0.x, FF3.6.x but would no longer need to support FF3.5.x.
In that scenario, we might save that branch support for Firefox, but as Thunderbird and SeaMonkey will ship major releases based on Mozilla 1.9.1 within the next few weeks, we will need to support that branch in some way for some time anyhow, at least until those projects are ready to release something on a post-1.9.1 branch.
Mike Beltzner wrote: > On 10/15/2009 8:05 PM, Axel Hecht wrote: >> we've discussed before if we're doing the update from 3.5 to 3.6 as >> minor update.
...
> // Compatibility concerns
> Add-on compatibility is one of the large reasons why users do not move > from one version to another.
Now I am responding for the Firebug users rather than the Firebug developers.
Based on the past Firefox update approach, Firebug versions span two versions of Firefox: # Firebug 1.3 works with Firefox 3.0 and Firefox 2.0 # Firebug 1.4 works with Firefox 3.0 and Firefox 3.5 # Firebug 1.5 will work with Firefox 3.5 and Firefox 3.6
Firebug 1.4 is not compatible with Firefox 3.6 and won't be. So if 3.6 overwrites 3.5, Firebug 1.4 users will have to manually install Firefox 3.5.
The existence, testing, and compatibility of Firebug 1.5 is not relevant: these are Firebug 1.4 users and they will remain so until 1.5 has been out for many months. I have no idea how many people fit this category.
Firebug may be unusual, but in general the model that an extension is "compatible" with a version of Firefox does not work for us. Firebug is not a stationary fixture. We are changing (maybe faster than Firefox?) and we are very dependent on Firefox details. So even though Firefox team may view 3.5 -> 3.6 as minor, Firebug users will see it as major because they will be forced to adopt Firebug 1.5 at the same time.
> Scenario#1: > =========== > - on day of release of FF3.6.0, existing FF3.5.x users get a *minor* > update to FF3.6.0. > ...which means: > -- by using that minor upgrade dialog, user will find it harder to see > what changed until *after* installation
The entire point is that from a user perspective, little to no visible change will be present. Firefox will just be faster, more stable, and do a bit more in terms of web technology support. The only visible UI feature is support for a new type of theme, and that's not anything that presents itself in the primary UI.
Further, the major update dialog doesn't really give users any prescience into what the changes will be. The billboard we display is usually a recap of our primary marketing messages.
> -- users could be surprised by change in functionality (and UI?) when > they are used to only get small security fixes in a minor update > scenario. Downgrading is tricker? Would this surprise encourage > users to > turn off getting Firefox updates in future?
I'll keep saying it: there is no change in functionality. There is no visible change in UI.
> -- concerns about new problems discovered only after a lot of users > were > broken. Bad user experience. > -- mozilla will need to support one less branch. We still support > FF3.0.x, FF3.6.x but would no longer need to support FF3.5.x.
> Scenario#2: > =========== > - on day of release of FF3.6.0, existing FF3.5.x users get a prompted > *major* update. > ...which means: > -- upgrade dialog box has details of what will change. > -- users would not be surprised by change. > -- mozilla will need to support FF3.5 in addition to FF3.0.x, FF3.6.x. > -- concerns about new problems discovered only after a lot of users > were > broken. Bad user experience. > -- maybe less risk if we did this with FF3.6.1 instead?
I think you're overestimating the value of the upgrade box, as mentioned above. Also, this scenario fails to take into account that once we offer Firefox 3.6 as a major update, it becomes increasingly difficult (from a product presentation and messaging point of view) to offer it as a minor update in the future. The major update offer clearly provides the user a choice of whether or not they want to upgrade, and I'd be hard pressed to remove that option later.
> Scenario#3: > =========== > - on day of release of FF3.6.0, existing FF3.5.x users are not > prompted > to update, but users can choose to manually-only *major* update to > FF3.6.0. This is just like we did for the release of FF3.5.0, was > quite > used for 35-45% of upgrades, and we know our infrastructure can do > this > safely. > - early adopters would pro-actively upgrade to 3.6.0 > - we monitor for crash reports or any problems. > -- if there are no red flags after a week, we can proceed to: > -- start a *gradual* rollout of idle-background *major* update. AUS > allows us to gradually un-throttle MU prompts to idle-users, so only a > subset of users would see the prompt. Maybe start with 10%? > - if there are no red flags after another week, we can raise to 20%, > then 50%, then 100%. > ...which means: > -- we have time to react to surprise bugs if needed > -- we can reverse throttling if needed to limit exposure to surprise > bugs as they are discovered. > -- user-visible upgrade dialog box has details of what changed. > -- we offer MU more quickly then we did for FF3.0->FF3.5 > -- mozilla will need to support FF3.5 in addition to FF3.0.x, FF3.6.x.
I think that this is mostly the same as what I wrote here:
>> This also hasn't been fully fleshed out, but it's possible to release >> Firefox 3.6 and not update Firefox 3.5 users immediately. In the past >> we've avoided doing that for bandwidth preservation reasons alone, >> and >> we may wish to continue with that policy.
or at least I intended to infer what you've written above. There's an additional cost of producing and localizing the "billboard" for the major update dialog, which takes time and localization.
> It might be worth noting that at Nokia, we hit a huge rough patch > trying to update from 1.9.2pre to 1.9.2, namely, another group had a > NPAPI plugin using the XPCOM Scriptable Plugin solution (obsolete pre > Firefox 1.0, but they didn't ask us, and really refuse to let us > manage them or their priorities). For anyone using this plugin, the > update isn't a minor update, since it breaks it entirely (this has > effectively blocked our ability to upgrade to a remotely safe version > of Gecko for MicroB on the N900).
This change occurred between 1.9.2a1 and 1.9.2b1? Or between 1.9.1.x and 1.9.2? I'd be curious to know which bugs changed this and why.
> From a certain perspective, I'd actually encourage you to upgrade > everyone to FF3.6 so that these people will understand that their > priorities are backwards. OTOH, if you actually care about people who > might be using this plugin, you might want to do something else.
> Firebug 1.4 is not compatible with Firefox 3.6 and won't be. So if > 3.6 overwrites 3.5, Firebug 1.4 users will have to manually install > Firefox 3.5.
Why manually instead of using the Add-on update system which allows you to publish information about compatibility updates and then have Firefox automatically install that update when it moves to the new version?
> Firebug may be unusual, but in general the model that an extension > is "compatible" with a version of Firefox does not work for us. > Firebug is not a stationary fixture. We are changing (maybe faster > than Firefox?) and we are very dependent on Firefox details. So even > though Firefox team may view 3.5 -> 3.6 as minor, Firebug users will > see it as major because they will be forced to adopt Firebug 1.5 at > the same time.
Have you seen a large amount of developer reluctance to move from Firebug 1.4 to 1.5? Is it not a superior product? Are you not eager to have more users on the newer version? I think I'm a little confused by the implication that being forced to move to Firebug 1.5 will be an issue of contention for users.
Further, I would expect that many web developers are (sadly) used to retaining an older version of Firefox on their system to use with Firebug do to the lag we'd previously encountered between a Firefox and compatible Firebug release.
> Mike Beltzner wrote: >> Finally, throughout the development cycle of Firefox 3.6 we have been >> taking fewer invasive changes which should make it easier to reason >> about and test for compatibility concerns.
> But I think enough that it upset quite a number of people if we > silently force those changes upon them.
That's a fine thing to say, but in order to factor that into the analysis it would be helpful if you could expand on what you consider to be "quite a number of people" (note that we're now in a world where upsetting 1M users is less than 1% of our total audience), as well as what the cost of upsetting those users (f.e., OSX 10.6 broke a ton of 3rd party applications, yet few people ended up downgrading, instead transferring their expectations onto those application vendors to issue compatibility updates).
I think to reason properly about those things, it would help to know which changes in mozilla-1.9.2 / Firefox 3.6 you believe will cause users to be upset.
> This change occurred between 1.9.2a1 and 1.9.2b1? Or between 1.9.1.x and > 1.9.2? I'd be curious to know which bugs changed this and why.
It changed before 1.9.2a1. It is part of restructuring the plugin host to make the code cleaner, as well as preparation for Electrolysis and perhaps changing the XPCOM ABI.
> Based on what I've seen so far, 3.6b1 resembles previous 3.Xb1 in that > it will be (from my perspective) not significantly better than nightly > builds. It seems that a lot of time and energy goes into the b1, but it > falls short. I don't think this is just "duh". Rather I think the > nightly build infrastructure is really excellent as far as it goes, but > whatever is going on after that does not add enough value at the b1 > stage. The b1 becomes an instant reject because some few aspects don't > work for extensions which are vital to the beta user community. In > future, consider putting less energy into b1 and more energy into > shortening the distance between b1 and b2 where extensions have had a > chance to get fixes in. Perhaps even invent a "bx" release, explicitly > for this purpose.
And I don't understand what you mean by "significantly better than nightly builds". Typically the beta is a nightly build, done after we've fixed all the bugs we consider serious enough to block a beta. Are you saying there are bugs that should have blocked the beta, but didn't (and can you provide examples)?
It seems like you're saying "beta1 is always bad because extensions are broken". How can we help extension authors to test with the nightly builds so that they aren't broken by the time we get to beta?
There is no planned beta2. The next milestone is the first release candidate.
> In that scenario, we might save that branch support for Firefox, but as > Thunderbird and SeaMonkey will ship major releases based on Mozilla > 1.9.1 within the next few weeks, we will need to support that branch in > some way for some time anyhow, at least until those projects are ready > to release something on a post-1.9.1 branch.
Although SeaMonkey and Thunderbird are basing current work on 1.9.1, I think that it would be bad for the Mozilla community and future web work to maintain yet another branch, which is one of the driving forces behind making Firefox 3.6 a minor update. This means that the seamonkey and thunderbird developers would end up being responsible for backporting security patches to 1.9.1, or updating your code to work in 1.9.2. Either way, I don't think that core gecko hackers and the security teams should be signed up to maintain another branch.
Mike Beltzner wrote: > On 2009-10-21, at 11:45 AM, John J. Barton wrote:
>> Firebug 1.4 is not compatible with Firefox 3.6 and won't be. So if 3.6 >> overwrites 3.5, Firebug 1.4 users will have to manually install >> Firefox 3.5.
> Why manually instead of using the Add-on update system which allows you > to publish information about compatibility updates and then have Firefox > automatically install that update when it moves to the new version?
I don't think the Add-on system will automatically downgrade Firefox if an Add-on is not compatible. These are Firebug 1.4 users, not Firebug users. That's my point.
>> Firebug may be unusual, but in general the model that an extension is >> "compatible" with a version of Firefox does not work for us. Firebug >> is not a stationary fixture. We are changing (maybe faster than >> Firefox?) and we are very dependent on Firefox details. So even though >> Firefox team may view 3.5 -> 3.6 as minor, Firebug users will see it >> as major because they will be forced to adopt Firebug 1.5 at the same >> time.
> Have you seen a large amount of developer reluctance to move from > Firebug 1.4 to 1.5?
I can't answer since 1.5 does not yet exist. Numerically the vast majority of Firebug users upgrade when we update AMO. But the long tail is critical because it contains the majority of developers for large sites.
> Is it not a superior product?
By which criteria? Of course *I* think 1.5 is way better than 1.4. But in one important category 1.5 is certain to be inferior: successfully used by many users on many different sites.
> Are you not eager to > have more users on the newer version?
Of course. My eagerness is not at issue however. ;-)
> I think I'm a little confused by > the implication that being forced to move to Firebug 1.5 will be an > issue of contention for users.
And we're a little confused by a Firefox 3.6 that can't decide if it is 3.6 or 3.5.5. If 3.6 is really minor, release it as 3.5.5. Else, well then it's not minor after all.
> Further, I would expect that many web developers are (sadly) used to > retaining an older version of Firefox on their system to use with > Firebug do to the lag we'd previously encountered between a Firefox and > compatible Firebug release.
Yes, this is exactly the issue we are discussing! But you are viewing "compatible" as some kind of simple one-time test. But web devs view "compatible" as "works 100% on my problems". We have no way of providing that kind of compatibility other than by having users try it and report bugs. That is why the overlapping version solution works for us. Some of the community moves on to the new version, we fix bugs, then the laggards join. By that time we are on the next version.
Another factor may be less important because of changes in the browser arena. In the past web devs lagged because their users lagged: they had to support back level browsers so they needed dev tools on back level browsers.
Despite all of this I want to say I support shorter release cycles in Firefox. Firebug shortened it cycle so we can be in sync. I'm just trying to report how Firebug users may see this idea of pushing 3.6 on top of 3.5 too soon. As I said before, I think the b1 should be sooner and their should be a time when both 3.5 and 3.6 exist to support transitions.
Mike Beltzner wrote: > On 2009-10-21, at 11:24 AM, Robert Kaiser wrote:
>> Mike Beltzner wrote: >>> Finally, throughout the development cycle of Firefox 3.6 we have been >>> taking fewer invasive changes which should make it easier to reason >>> about and test for compatibility concerns.
>> But I think enough that it upset quite a number of people if we >> silently force those changes upon them.
> That's a fine thing to say, but in order to factor that into the > analysis it would be helpful if you could expand on what you consider > to be "quite a number of people" (note that we're now in a world where > upsetting 1M users is less than 1% of our total audience),
I think the concern Kiro is expressing here are the pool of users in the 'high performance sedan' analogy I posted earlier. When we upgrade them we should do it at a time where 10 million browser enthusiast have weighed in to tell us the release is ready for broader distribution. Right now we are set to go from several hundred thousand or a million to many tens of millons in a matter of days or weeks, and its indiscriminate updating of both browser enthusiasts and browser conasures that are expecting the highest reliability.
> as well as what the cost of upsetting those users (f.e., OSX 10.6 > broke a ton of 3rd party applications, yet few people ended up > downgrading, instead transferring their expectations onto those > application vendors to issue compatibility updates).
downgrading an OS is hard. downgrading or swtiching to another browser is also hard for many but relatively easier.
> I think to reason properly about those things, it would help to know > which changes in mozilla-1.9.2 / Firefox 3.6 you believe will cause > users to be upset.
Its the unknown things that we learn in extended beta(s), rc(s), and early part of releases that could be the biggest concern. compression of that part of the development cycle means higher risk.
Mike Beltzner wrote: > On 2009-10-21, at 6:36 AM, John O'Duinn wrote:
>> Scenario#1: >> =========== >> - on day of release of FF3.6.0, existing FF3.5.x users get a *minor* >> update to FF3.6.0. >> ...which means: >> -- by using that minor upgrade dialog, user will find it harder to see >> what changed until *after* installation
> The entire point is that from a user perspective, little to no visible > change will be present. Firefox will just be faster, more stable, and do > a bit more in terms of web technology support. The only visible UI > feature is support for a new type of theme, and that's not anything that > presents itself in the primary UI.
> Further, the major update dialog doesn't really give users any > prescience into what the changes will be. The billboard we display is > usually a recap of our primary marketing messages.
...which includes the outline of top new features, yes?
>> -- users could be surprised by change in functionality (and UI?) when >> they are used to only get small security fixes in a minor update >> scenario. Downgrading is tricker? Would this surprise encourage users to >> turn off getting Firefox updates in future?
> I'll keep saying it: there is no change in functionality. There is no > visible change in UI.
I'd accept "no known change in functionality", but yes, ok.
>> -- concerns about new problems discovered only after a lot of users were >> broken. Bad user experience. >> -- mozilla will need to support one less branch. We still support >> FF3.0.x, FF3.6.x but would no longer need to support FF3.5.x.
>> Scenario#2: >> =========== >> - on day of release of FF3.6.0, existing FF3.5.x users get a prompted >> *major* update. >> ...which means: >> -- upgrade dialog box has details of what will change. >> -- users would not be surprised by change. >> -- mozilla will need to support FF3.5 in addition to FF3.0.x, FF3.6.x. >> -- concerns about new problems discovered only after a lot of users were >> broken. Bad user experience. >> -- maybe less risk if we did this with FF3.6.1 instead?
> I think you're overestimating the value of the upgrade box, as mentioned > above. Also, this scenario fails to take into account that once we offer > Firefox 3.6 as a major update, it becomes increasingly difficult (from a > product presentation and messaging point of view) to offer it as a minor > update in the future. The major update offer clearly provides the user a > choice of whether or not they want to upgrade, and I'd be hard pressed > to remove that option later.
I agree that once we start offering FF3.5->FF3.6 as a MU, its hard to later change our minds and offer FF3.5->FF3.6 as a minor update. I think we would stay one way or another.
>> Scenario#3: >> =========== >> - on day of release of FF3.6.0, existing FF3.5.x users are not prompted >> to update, but users can choose to manually-only *major* update to >> FF3.6.0. This is just like we did for the release of FF3.5.0, was quite >> used for 35-45% of upgrades, and we know our infrastructure can do this >> safely. >> - early adopters would pro-actively upgrade to 3.6.0 >> - we monitor for crash reports or any problems. >> -- if there are no red flags after a week, we can proceed to: >> -- start a *gradual* rollout of idle-background *major* update. AUS >> allows us to gradually un-throttle MU prompts to idle-users, so only a >> subset of users would see the prompt. Maybe start with 10%? >> - if there are no red flags after another week, we can raise to 20%, >> then 50%, then 100%. >> ...which means: >> -- we have time to react to surprise bugs if needed >> -- we can reverse throttling if needed to limit exposure to surprise >> bugs as they are discovered. >> -- user-visible upgrade dialog box has details of what changed. >> -- we offer MU more quickly then we did for FF3.0->FF3.5 >> -- mozilla will need to support FF3.5 in addition to FF3.0.x, FF3.6.x.
> I think that this is mostly the same as what I wrote here:
>>> This also hasn't been fully fleshed out, but it's possible to release >>> Firefox 3.6 and not update Firefox 3.5 users immediately. In the past >>> we've avoided doing that for bandwidth preservation reasons alone, and >>> we may wish to continue with that policy.
err... reads quite different, but maybe you meant "not update all Firefox"? Anyway, if you are asking "is scenario#3 possible", I confirm this is mechanically possible.
Scenario#3 is something we can do mechanically, and feels like an improvement over what we did for FF3.0->FF3.5.
While it feels so long ago now, I note that FF3.5 was the first time we had MU available on the FF3.5 release day. And even then, it was for manual-check-for-update only. Having FF3.5->FF3.6 MU available on FF3.6 release day for *all* users would be a big incremental step forward.
Lets try this before we start try crossing-release-branches-with-minor-updates?
> There's an > additional cost of producing and localizing the "billboard" for the > major update dialog, which takes time and localization.
Mike Beltzner wrote: > On 2009-10-21, at 11:24 AM, Robert Kaiser wrote:
>> Mike Beltzner wrote: >>> Finally, throughout the development cycle of Firefox 3.6 we have been >>> taking fewer invasive changes which should make it easier to reason >>> about and test for compatibility concerns.
>> But I think enough that it upset quite a number of people if we >> silently force those changes upon them.
> That's a fine thing to say, but in order to factor that into the > analysis it would be helpful if you could expand on what you consider to > be "quite a number of people" (note that we're now in a world where > upsetting 1M users is less than 1% of our total audience), as well as > what the cost of upsetting those users (f.e., OSX 10.6 broke a ton of > 3rd party applications, yet few people ended up downgrading, instead > transferring their expectations onto those application vendors to issue > compatibility updates).
Umm, true, even though it's always hard to wrap my mind around that, knowing that the product I'm managing has about 1M as the *total* user base ;-)
> I think to reason properly about those things, it would help to know > which changes in mozilla-1.9.2 / Firefox 3.6 you believe will cause > users to be upset.
Where do we have those nice lists of things changed in the release? We could try to sort out from there what has which impact.
IIRC, we for example removed the OJI support, which means that a number of users will probably end up with broken Java unless they upgrade that as well. timeless has already mentioned XPCOM Scriptable Plugin support, there might still be some plugins out there that use it, which also would just stop working with a "minor" upgrade if we do that for 3.5->3.6.
I don't think being unable to import download history from Mozilla 1.8.* and older is much of an issue for Firefox right now, but it's one of the reasons why SeaMonkey needs to stick with 1.9.1 for a few months, where SeaMonkey 1.x users still can migrate all their data.
There might be other things that pop up when we go through a changelog piece by piece, as my product is only concentrating on 1.9.1 for now, I'm far from an expert on 1.9.2 changes.
> IIRC, we for example removed the OJI support, which means that a number > of users will probably end up with broken Java unless they upgrade that > as well.
Benjamin Smedberg wrote: > On 10/21/09 11:29 AM, Robert Kaiser wrote:
>> In that scenario, we might save that branch support for Firefox, but as >> Thunderbird and SeaMonkey will ship major releases based on Mozilla >> 1.9.1 within the next few weeks, we will need to support that branch in >> some way for some time anyhow, at least until those projects are ready >> to release something on a post-1.9.1 branch.
> Although SeaMonkey and Thunderbird are basing current work on 1.9.1, I think > that it would be bad for the Mozilla community and future web work to > maintain yet another branch, which is one of the driving forces behind > making Firefox 3.6 a minor update. This means that the seamonkey and > thunderbird developers would end up being responsible for backporting > security patches to 1.9.1, or updating your code to work in 1.9.2. Either > way, I don't think that core gecko hackers and the security teams should be > signed up to maintain another branch.
Gecko and security teams (as well as Linus distro teams) are still doing some maintenance for 1.8 branch at this time, and have been doing so for quite some time. Would you prefer to have them prolong maintaining that for two or three months while we make sure Thunderbird and SeaMonkey can ship from 1.9.2 or would you prefer us releasing on 1.9.1 now and have them maintaining that branch for a bit?
> Gecko and security teams (as well as Linus distro teams) are still doing > some maintenance for 1.8 branch at this time, and have been doing so for > quite some time. Would you prefer to have them prolong maintaining that > for two or three months while we make sure Thunderbird and SeaMonkey can > ship from 1.9.2 or would you prefer us releasing on 1.9.1 now and have > them maintaining that branch for a bit?
Those are not the only options: you could release off of 1.9.1 now, and do a minor update in TB3.0.2 switching to the 1.9.2 gecko base. That is the solution I recommend.
I believe Mozilla as a community should focus on moving the web forward. If TBird/SeaMonkey are going to cause lots of friction and maintenance, as a community we should lower our support, rather than accepting the security/maintenance friction of another active security branch.
