Coupled Train Model

Showing 1-85 of 85 messages
Coupled Train Model Alex Keybl 10/18/13 11:10 AM
The main purpose of Aurora as a channel is to get early feedback on new Firefox versions with users that better represent our Beta/Release populations. Aurora users knowingly accept a lower quality bar than Beta/Release, as well as more frequent updates.

Aurora is a fantastic tool, but it's my team's opinion that it doesn't make great use of the full six weeks allotted to it in each development cycle. We find most major Aurora issues in the first week or so after the merge, and critical Beta-blocking issues are typically resolved within days thereafter. We've verified this premise by taking a look at bugs tracked for Firefox 24, fixed in the Aurora timeframe.

We have an informal saying in Release Management that we should always "get new code in front of the most users as soon as possible". Given that goal, I'd like us to spend even more of the 18 weeks with the much larger Beta population, since that population is the most applicable to Release in diversity and quality on Aurora is already fairly high. Here's my proposed "Coupled Train Model" (thanks to curtisk for the naming):

* change the desktop/android train model to have two 9-week cycles rather than three 6-week cycles (still 18 weeks start to finish)
* enable Aurora updates the day after release, rather than at the end of the week
* merge mozilla-aurora to mozilla-beta as soon as critical quality issues have been resolved on both Aurora and Release (1-2 weeks after that)
** we'd utilize the 1-2 week overlap to push dot releases to the Beta population prior to Release (bonus functionality!)
* continue to have certain features only enabled on Aurora and lower
* continue to use mozilla-aurora and the Aurora population as a method of testing fixes prior to uplift to mozilla-beta
* allow developers to focus on two pre-release Gecko versions at a time, rather than three

This proposal sees the Aurora population as a shorter, intermediate step towards Beta testing rather than an equally weighted phase of development (in terms of time). A chart explaining the Coupled Train Model can be found at https://pbs.twimg.com/media/BV7r23WCMAAROMP.png:large

Things we're still considering, and we're meeting separately on:

* a start date - current proposal is starting with Firefox 30 in the new year
* a new string freeze date - current proposal is upon the merge to Beta, since features that aren't ready for release will be backed out by that point
* an API freeze date, for add-on/plugin authors - still need to determine a hard date, but somewhere shortly after the merge to Beta
* when (and from where) to branch future FxOS releases
* security update frequency and the bar for a security dot release (given platform updates every 9 weeks)
* how we would want to communicate the Aurora/Beta "releases" (perhaps together?)

Thanks to all those who have already provided feedback (and early support) at the Open Sessions on Releases in Santa Clara and Toronto. I'd love to hear even more feedback from those community members who weren't able to attend.

-Alex
Re: Coupled Train Model Boris Zbarsky 10/18/13 11:39 AM
On 10/18/13 2:10 PM, Alex Keybl wrote:
> The main purpose of Aurora as a channel is to get early feedback on new Firefox versions with users that better represent our Beta/Release populations.

A major purpose of Aurora (that I think we've totally failed to
effectively publicize) is to provide a less-scary-than-nightly option
for web developers who want to experiment with new functionality that's
not enabled by default yet and provide feedback on it.  This is why
various experimental stuff is enabled on Aurora.

In the new model, will Aurora still be usable for this purpose?  I guess
it will, since after 2 weeks or so Aurora will basically be "Beta with
the experimental stuff turned on"?

-Boris
Re: Coupled Train Model Alexander Keybl 10/18/13 1:33 PM
Correct, there can continue to be Aurora specific features disabled on Beta.

-Alex
_______________________________________________
dev-planning mailing list
dev-pl...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-planning

Re: Coupled Train Model Nicolas B. Pierron 10/18/13 2:17 PM
On 10/18/2013 11:10 AM, Alex Keybl wrote:
 > * change the desktop/android train model to have two 9-week cycles rather
 > than three 6-week cycles (still 18 weeks start to finish)

So, after 9 weeks of nightly (instead of 6 weeks), we will merge into
aurora?  Do we really expect that we would be able to fix 1.5 times more
code in the same period of time on aurora?

Also, B2G development is currently synchronized with even versions of Gecko.
  What would be the impact there? I don't think we want to switch B2G to a
18 weeks cycle, can we switch to a 9 weeks cycle?

On 10/18/2013 11:10 AM, Alex Keybl wrote:
 > * merge mozilla-aurora to mozilla-beta as soon as critical quality issues
 > have been resolved on both Aurora and Release (1-2 weeks after that)

What does «as soon as critical quality issues have been resolved» implies?
Does that mean that the overlap period would be variable period of time and
we shift the train only when things are good enough?

When I look at the diagram, I wonder what would be the advantage of anybody
to use Aurora if this is just to have all the bugs & features 1-2 weeks
before every beta users.  Wouldn't that decrease the remaining Aurora users,
in which case this grace period for beta would be useless as we would have
either nightly users or beta users?

--
Nicolas B. Pierron
Re: Coupled Train Model Zack Weinberg 10/18/13 2:25 PM
On 2013-10-18 2:39 PM, Boris Zbarsky wrote:
> On 10/18/13 2:10 PM, Alex Keybl wrote:
>> The main purpose of Aurora as a channel is to get early feedback on
>> new Firefox versions with users that better represent our Beta/Release
>> populations.
>
> A major purpose of Aurora (that I think we've totally failed to
> effectively publicize) is to provide a less-scary-than-nightly option
> for web developers who want to experiment with new functionality that's
> not enabled by default yet and provide feedback on it.  This is why
> various experimental stuff is enabled on Aurora.

Periodic reminder that as long as we don't have one-checkbox
side-by-side installation of multiple channels (simultaneously runnable,
in separate profiles, sync hooked up automatically), web developers
cannot be bothered even with Beta, let alone Aurora.

zw
Re: Coupled Train Model Gavin Sharp 10/18/13 3:14 PM
On Fri, Oct 18, 2013 at 2:17 PM, Nicolas B. Pierron
<nicolas....@mozilla.com> wrote:
> So, after 9 weeks of nightly (instead of 6 weeks), we will merge into
> aurora?  Do we really expect that we would be able to fix 1.5 times more
> code in the same period of time on aurora?

It's not "the same period of time on aurora" - it's 9 weeks spread
across beta/aurora. So for stabilization, instead of 6 weeks on Aurora
+ 6 weeks on Beta, we get 2 weeks on Aurora and 7 weeks on
Aurora+Beta.

Gavin
Re: Coupled Train Model Axel Hecht 10/19/13 4:28 AM
On 10/18/13 8:10 PM, Alex Keybl wrote:
> * a new string freeze date - current proposal is upon the merge to Beta, since features that aren't ready for release will be backed out by that point
That's not really enough detail to comment on the l10n aspects of your
proposal. I'd like you to explain how you're suggesting to get to a
localized release.

A bit of data as food for thought: On Firefox 24, 70 teams signed off,
53 of which were on aurora or earlier work. 17 locales only caught up on
beta, while 13 signed off on additional fixes. That could just be the
aurora-beta merge commit, though, too.

Axel
Re: Coupled Train Model Archaeopteryx 10/19/13 3:50 PM
Questions which came to my mind:

* How will we attract users to Aurora if they get the same branch for 7
of 9 weeks on Beta?

* Developers will have 9 weeks time for bug fixing/polishing/making the
product end user ready and 9 weeks for developing, while nowadays they
have 12 weeks for "fixing" features developed in 6 weeks. This could be
problematic directly after the change because they aren't yet used to it.

* An API freeze shortly after the beta merge when parts of API changes
are still dev-doc-needed will make add-on compatibility changes for
add-on developers very hard. Any ideas how to improve that situation
(the timeframe for add-on developers has also been halved)?

* L10n:

** If there will be no fixed date when localization can start, most
localizers will start working at a moment which would have also worked
before.

** Let's say two weeks on Aurora, that leaves 7 weeks on Beta.
Localizers have to finish the localization with a signoff more than a
week before release. Because that's a Monday and the signoff request
usually doesn't get reviewed earlier on that Monday or the weekend
before, that's 1.5 weeks less for localization. So there are 5.5 weeks
left for localization. Given that localizations on Beta have to be
reviewed to get into production (not required on Aurora), it will be
hard to discover issues and collect feedback in that time frame before.
At the moment, localizers have 11 weeks from start to end.

Thank you in advance
Sebastian
Re: Coupled Train Model Gavin Sharp 10/20/13 12:05 PM
On Sat, Oct 19, 2013 at 3:50 PM, Archaeopteryx
<archae...@coole-files.de> wrote:
> * How will we attract users to Aurora if they get the same branch for 7 of 9
> weeks on Beta?

Attracting users to Aurora isn't really a goal in and of itself (it's
only a goal insofar as it helps us ship better software). This
proposal seems like the result of confronting the reality that we have
not been able to maintain a useful Aurora differentiation for users,
so we might as well try a different approach and (mostly) combine it
with beta.

Gavin
Re: Coupled Train Model Nicolas B. Pierron 10/20/13 4:53 PM
On 10/18/2013 11:10 AM, Alex Keybl wrote:
 > We find most major Aurora issues in the first week or so after the merge,
 > and critical Beta-blocking issues are typically resolved within days
 > thereafter.

If I understand correctly, we are trying to optimize the train model based
on this fact.

 > We have an informal saying in Release Management that we should always
 > "get new code in front of the most users as soon as possible".

The problem is that Aurora does not have enough users.  Currently we have
almost the same number of Aurora users as Nightly users, which is too low
compared to Beta.

The "Coupled Train Model" suggests to give the changes sooner to Beta users
by only making use of Aurora for a few weeks.  After these few weeks the
only difference would be features which are enabled in one but not the other.

I fear that doing so will reduce even more the number of Aurora users by
making Aurora less attractive.  Thus this would emphasize the original
problem, by reducing the number of user on Aurora. A consequence of reducing
the number of Aurora users would be to augment the time needed for finding
"most major Aurora issues".

 > We have an informal saying in Release Management that we should always
 > "get new code in front of the most users as soon as possible".

Why not using this informal saying, to increase our number of users on
Aurora?  One model KaiRo and I came up with while trying to answer the same
question was to use a "Wagon Train Model" [1].

The idea being to merge multiple time from Nightly into Aurora, and keeping
the rest unchanged.  This way, instead of having Aurora useful for only the
first 1-2 weeks, we can make a full use of it by finding major issues sooner.

These 2 models are not incompatible, as you pointed out in a private reply.
  Still, I think the "Coupled Train Model" does not try to solve the root of
the problem, which is that we do not have enough Aurora users.  And worse,
it might emphasize the problem.

On the other hand, the "Wagon Train Model" attempt to address the problem by
making Aurora more attractive, and thus increasing the number of Aurora users.

[1]
https://docs.google.com/spreadsheet/ccc?key=0AkXdfV25b2n5dFFfUzFqcHZnaVVVbWw1LXRXVXNqTFE&usp=sharing

--
Nicolas B. Pierron

Re: Coupled Train Model Alex Keybl 10/21/13 6:16 AM
We'll have a meeting soon to float ideas and come to an agreement with you. I've had early conversations with chofmann, who noted that the most it's most important to give localizers a stable string base and enough time to translate. We're working hard to make sure we make everybody happy with the new tweaks.

-Alex
Re: Coupled Train Model Alex Keybl 10/21/13 6:18 AM
> * How will we attract users to Aurora if they get the same branch for 7 of 9 weeks on Beta?

Aurora as a population has not grown significantly over the last two years, but rather maintained. Note that Aurora will have some features that are disabled for Beta.

> * Developers will have 9 weeks time for bug fixing/polishing/making the product end user ready and 9 weeks for developing, while nowadays they have 12 weeks for "fixing" features developed in 6 weeks. This could be problematic directly after the change because they aren't yet used to it.

I do expect a bumpy first couple of cycles, but we feel it's for the best in the longterm.

> * An API freeze shortly after the beta merge when parts of API changes are still dev-doc-needed will make add-on compatibility changes for add-on developers very hard. Any ideas how to improve that situation (the timeframe for add-on developers has also been halved)?

Early feedback from add-on developers is that less frequent updates fits their dev schedule better. We'll have to figure out an API freeze date that works for both Mozilla and third parties.

-Alex

On Oct 19, 2013, at 6:50 PM, Archaeopteryx <archae...@coole-files.de> wrote:

> Questions which came to my mind:
>
> * How will we attract users to Aurora if they get the same branch for 7 of 9 weeks on Beta?
>
Re: Coupled Train Model Alex Keybl 10/21/13 6:21 AM
> Why not using this informal saying, to increase our number of users on Aurora?  One model KaiRo and I came up with while trying to answer the same question was to use a "Wagon Train Model" [1].

As you noted, any benefits it brings can be worked into this model in the future (more frequent merges to Aurora from Nightly). I'd really like to keep this thread on topic, if possible.

> These 2 models are not incompatible, as you pointed out in a private reply.  Still, I think the "Coupled Train Model" does not try to solve the root of the problem, which is that we do not have enough Aurora users.  And worse, it might emphasize the problem.

As Gavin mentioned, we're accepting the current state of populations and trying to make best use of them. We don't have any data to suggest that your proposed model will improve Aurora population size, but we do have data that suggests more time with the beta channel would yield more fixes in a release.

-Alex
Re: Coupled Train Model Axel Hecht 10/21/13 6:47 AM
I personally care about getting a great localized release channel out to
users. That's 60% of our users, and I want a great product for those. I
guess everyone else wants that, too.

On the way there, we currently have some vehicles/channels with a
variety of properties and social contracts. Most of those aren't well
defined. As far as I understand those, you're proposing to change at
least some of them.

I'd ask you to define what you think the various builds should be in
particular with l10n in mind, so that I don't have to speculate about
those, and put words in your mouth.

Right now, I think that calendars are a technical detail. Sure, we need
to make them useful, but I don't see how we'd do that without knowing
what the blocks on that calendar actually stand for.

Axel
Re: Coupled Train Model chris hofmann 10/21/13 8:10 AM
On 10/21/13 6:16 AM, Alex Keybl wrote:
> We'll have a meeting soon to float ideas and come to an agreement with you. I've had early conversations with chofmann, who noted that the most it's most important to give localizers a stable string base and enough time to translate.

I'll also add that what I think we discussed was that getting
localization work finished before moving to beta was a key part of the
work that needs to be done on aurora.   As Axel mentioned most of our
users don't use the English version.  It's actually 70%, so serving and
building larger beta populations around the world should be a key goal
to improving quality.

When you explained the idea to me a second time it was actually just an
effort to move in the most orderly and fast way to getting to a stable
beta, then moving that set of software to beta.  The idea was the we
were just going to try and get more disciplined about not taking
features into beta, and avoiding risky changes that could destabilize.
    The two weeks was just a target for trying to achieve this, not a
fixed time table.    I also think the "9-9" confuses things even further
as part of this proposal because it hints at things happening automatically.

I think we are better served by this model and we need to just develop a
good checklist for what constitues "finishing aurora" in terms of having
localizations "done",  getting and checking stability data, finishing
the testing of experimental features as borris mentioned, and anything
else the aurora cycle is still targeted to achieve.   Then when all
those things are checked off it will be time to move as quickly as
possible to beta.

Thats the kind of proposal that makes most sense to me, and helps us to
achieve more disapline in building the software.  I think that we
sometimes forget about thats what we are trying to achieve with the
train model.  Its better discipline that we are trying to add.

-chofmann
Re: Coupled Train Model Clint Talbert 10/21/13 9:52 AM
On 10/18/2013 11:10 AM, Alex Keybl wrote:
> * change the desktop/android train model to have two 9-week cycles rather than three 6-week cycles (still 18 weeks start to finish)
> * enable Aurora updates the day after release, rather than at the end of the week
> * merge mozilla-aurora to mozilla-beta as soon as critical quality issues have been resolved on both Aurora and Release (1-2 weeks after that)
> ** we'd utilize the 1-2 week overlap to push dot releases to the Beta population prior to Release (bonus functionality!)
> * continue to have certain features only enabled on Aurora and lower
> * continue to use mozilla-aurora and the Aurora population as a method of testing fixes prior to uplift to mozilla-beta
> * allow developers to focus on two pre-release Gecko versions at a time, rather than three
>
> This proposal sees the Aurora population as a shorter, intermediate step towards Beta testing rather than an equally weighted phase of development (in terms of time). A chart explaining the Coupled Train Model can be found at https://pbs.twimg.com/media/BV7r23WCMAAROMP.png:large
>
>
I could be confused here, but I don't see how this is going to work with
QA. The way I read the diagram, the two weeks of "fixing aurora" are
going to happen at the same time as "doing stabilization point releases
on beta for the new release".  I don't think we want to force two
conflicting priorities on the QA teams where they have to chose between
supporting release stabilization and ensuring that aurora has stabilized
to be uplifted into beta.

Furthermore, the b2g side of the story is really unclear. B2G is trying
hard to align with the train model, and I don't see how the b2g model
would adapt to this.

Clint
Re: Coupled Train Model Aki Sasaki 10/21/13 10:20 AM
This would be nice, as would an equivalent to the
extensions.checkCompatibility.nightly=false.  With this set, I can use
nightly with all my addons.  Without this set, most of my addons go away
and I can't get them back.  Aiui, Aurora and Beta don't have
equivalents, other than extensions.checkCompatibility.VERSION which need
to be updated every 6 weeks, so when Nightly goes kablooey I
automatically reach for Release.  With an
extensions.checkCompatibility.{aurora,beta}=false I'd be more likely to
use them.

Re: Coupled Train Model Alex Keybl 10/21/13 10:54 AM
> I could be confused here, but I don't see how this is going to work with QA. The way I read the diagram, the two weeks of "fixing aurora" are going to happen at the same time as "doing stabilization point releases on beta for the new release".  I don't think we want to force two conflicting priorities on the QA teams where they have to chose between supporting release stabilization and ensuring that aurora has stabilized to be uplifted into beta.

I think that's a very narrow view of QA's bandwidth. Rarely do we have to choose between one investigation or another. We maintain 4 branches (3 pre-release), mind you. If we did have to choose, we'd of course prioritize Release though.

> Furthermore, the b2g side of the story is really unclear. B2G is trying hard to align with the train model, and I don't see how the b2g model would adapt to this.

That would be our work, so don't worry about us undoing it. The story is unclear because we haven't made final plans yet.

-Alex
Re: Coupled Train Model Jeff Griffiths 10/21/13 1:24 PM
This is a point we keep coming back to. If there aren't enough users on
Aurora to be useful in terms of testing or bug discovery I would love to
be able to use it to target web developers specifically ( we already
recommend it ) along with some developer-centric features. This could be
as simple as:

1) prompt the user on first-run of Aurora if they want a 'developer'
profile.

2) afterwards, only launch Aurora with the Aurora profile.

I like the automagic sync idea only to a point - developers are going to
tend to have different configurations / extensions / etc for a
dev-centric browser.

$0.02, Jeff
Re: Coupled Train Model anthony....@gmail.com 10/21/13 2:29 PM
On Monday, October 21, 2013 10:54:44 AM UTC-7, Alex Keybl wrote:
> > I could be confused here, but I don't see how this is going to work with QA. The way I read the diagram, the two weeks of "fixing aurora" are going to happen at the same time as "doing stabilization point releases on beta for the new release".  I don't think we want to force two conflicting priorities on the QA teams where they have to chose between supporting release stabilization and ensuring that aurora has stabilized to be uplifted into beta.
>
>
>
> I think that's a very narrow view of QA's bandwidth. Rarely do we have to choose between one investigation or another. We maintain 4 branches (3 pre-release), mind you. If we did have to choose, we'd of course prioritize Release though.

I cannot nor will I speak for the entire QA org; the following is purely my personal opinion. I'm not yet convinced that Aurora and the current train model are broken. I suspect it's how we are using them (or not) that is broken.

In my experience it's taken nearly everything Desktop QA has to keep up with the pace of uplifts and testing of Betas. I personally believe that QA should be living in Aurora but the current state of things does not afford us that opportunity. To get there we need better automation, tools for community involvement, and stricter, more clearly defined rules governing uplifts (especially on Beta).

I do not see how this proposal benefits the quality/stability of the product(s) we ship and I fear it might impose an even greater challenge on an already stretched QA team.
Re: Coupled Train Model Karl Tomlinson 10/21/13 3:10 PM
On Fri, 18 Oct 2013 14:10:52 -0400, Alex Keybl wrote:

> Aurora is a fantastic tool, but it's my team's opinion that it
> doesn't make great use of the full six weeks allotted to it in
> each development cycle. We find most major Aurora issues in the
> first week or so after the merge, and critical Beta-blocking
> issues are typically resolved within days thereafter. We've
> verified this premise by taking a look at bugs tracked for
> Firefox 24, fixed in the Aurora timeframe.
>
> We have an informal saying in Release Management that we should
> always "get new code in front of the most users as soon as
> possible". Given that goal, I'd like us to spend even more of
> the 18 weeks with the much larger Beta population, since that
> population is the most applicable to Release in diversity and
> quality on Aurora is already fairly high.

The benefits that I see from the proposal are:

1. Increase number of weeks of beta testing from 6 to hopefully 7.

2. Reduce time between m-c landing and release from 12-18 down to
   9-18.  Assuming evenly distributed landings the average
   reduction is 1.5 weeks.

The main risk/disadvantage is lowering the quality of aurora and
beta.  The Aurora channel would be almost as risky as Nightly at
merge time.  Beta would have 2 weeks of stabilization instead of 6.

My experience is that there are plenty of bugs on Nightly that
take more than 2 weeks to filter through triage, get a fix and
review.

Those fixes, and others that are resolved before 2 weeks,
currently have some time of testing before they need to appear on
Beta.  The new proposal either doesn't have testing time for these
fixes, or they don't make it in time for Beta users.

I would like to ask:

A. If most beta blocking issues are resolved within days, then how
   valuable is an extra week of beta testing?

B. Is it worth the cost of a lower quality beta?
   A lower quality beta would mean fewer users on that train, and
   therefore a narrower, less diverse, sample of testers and lower
   quality testing, IMHO more likely leading to a lower quality
   release product.
Re: Coupled Train Model Chris Peterson 10/21/13 4:01 PM
On 10/21/13 3:10 PM, Karl Tomlinson wrote:
> 1. Increase number of weeks of beta testing from 6 to hopefully 7.
>
> 2. Reduce time between m-c landing and release from 12-18 down to
>     9-18.  Assuming evenly distributed landings the average
>     reduction is 1.5 weeks.

Plus, the estimated 7 weeks of Beta assumes that, because we currently
stablize 6 weeks of Nightly development with 2 weeks on Aurora, we can
stablize 9 weeks of Nightly development with 2 weeks on Aurora.

Do we have reason to believe that Aurora stablization won't also
increase by 1.5x to 3 weeks? That would give us only 6 weeks of Beta
testing, i.e. no change from today's Beta duration, except with 3 weeks
of Aurora "bake time" (instead of 6) of 1.5x more code.


cpeterson
Re: Coupled Train Model Lawrence Mandel 10/21/13 4:30 PM
This is an interesting thought. IIRC, the original proposal saw Nightly remain at 6 weeks, Aurora drop to 3 weeks and beta move to 9 weeks. With the 9/9 cycle we appear to be optimizing for development time. Do we need/want additional development time in each cycle? Does an additional week on beta really warrant the effort required to make this change?

Lawrence
Re: Coupled Train Model Eric Rescorla 10/21/13 9:42 PM
Apologies if I have missed this, but I've been reading this thread and I
find myself wondering why we don't adopt a model more like Chrome's
(http://www.chromium.org/getting-involved/dev-channel#TOC-Channels)

- 6 weeks on Nightly ("Canary")
- 6 weeks on Beta
- 6 weeks on Stable

Chrome also has a "Dev" channel which is just the latest Nightly which
has received some testing. We could just rebrand Aurora in that model
and move forward. If we seriously think that Aurora isn't getting much
testing, anyway, wouldn't this be simplest?

Again, apologies if this has been discussed and rejected. If such a
discussion exists, I will gladly accept a pointer.

Thanks,
-Ekr





On Mon, Oct 21, 2013 at 4:30 PM, Lawrence Mandel <lma...@mozilla.com>wrote:
Re: Coupled Train Model Asa Dotzler 10/21/13 10:46 PM
On 10/18/2013 11:10 AM, Alex Keybl wrote:
> Thanks to all those who have already provided feedback (and early
> support) at the Open Sessions on Releases in Santa Clara and Toronto.
> I'd love to hear even more feedback from those community members who
> weren't able to attend.

I love that you had a session on this at the Summit and I'm sorry I
didn't get a chance to participate in that.

Was there any discussion at the Summit session about how the release
cadence maps not only to the efficient use of limited contributor
resources, but also how it impacts our end users?

I realize that you're not focused on the cadence of end user releases as
much as efficiently utilizing the pre-release channels, but if we're
going to shuffle the deck after two and a half years of rapid releases,
maybe it's worth bringing in more voices from the user-focused teams
like Product Management, Support/User Advocacy, UX/UR, Product
Marketing, etc.

My personal opinion is that our end users would love Firefox at least a
little bit more, possibly a lot more, if our release cadence was
quarterly.  But that's a very different discussion and perhaps one that
was already ruled out at the Summit sessions?

- A
Re: Coupled Train Model Ben Francis 10/22/13 7:34 AM
On Tue, Oct 22, 2013 at 5:42 AM, Eric Rescorla <e...@rtfm.com> wrote:

> I
> find myself wondering why we don't adopt a model more like Chrome's
> (http://www.chromium.org/getting-involved/dev-channel#TOC-Channels)
>
> - 6 weeks on Nightly ("Canary")
> - 6 weeks on Beta
> - 6 weeks on Stable


Are there any real blockers preventing us from doing this? If we think
Aurora isn't serving our needs well then removing it altogether in favour
of the 6/6/6 model Chrome uses would certainly equal out a competitive
disadvantage we currently have, with the time it takes to get a feature to
market. It also certainly helps with "getting new code in front of the most
users as soon as possible" and might provide an opportunity for us to
leverage the beta population more for testing and localisation, turning
more users into contributors and moving us closer to the goal of 1 million
Mozillians.

In addition to this it's been a huge effort to move to the 12 week train
model for Firefox OS and we're still only just getting started. It would be
challenging to change that again at the beginning of next year. A 12 week
cycle is already very aggressive from our partners' point of view but
"quarterly" releases is something we've talked quite loudly about as a
differentiator and is potentially disruptive in the mobile industry. A six
week heartbeat would mean we could continue running our 12 week Firefox OS
trains every other heartbeat.

In the long term I would love it if we could roll out silent 6 weekly
zero-rated Gecko updates to Firefox OS devices. We could split out many of
our Gaia apps (like media and productivity apps) into the Marketplace so
that partners have less to test and those apps can release directly to
users.

Let's continue to keep pushing the web forward, faster, on all fronts, but
keep everyone at Mozilla working to the same heartbeat.

My two cents.

Ben
Re: Coupled Train Model Stormy Peters 10/22/13 8:00 AM
On Mon, Oct 21, 2013 at 2:24 PM, Jeff Griffiths <jgrif...@mozilla.com>wrote:

>
>
> On 10/18/2013, 2:25 PM, Zack Weinberg wrote:
>
>> On 2013-10-18 2:39 PM, Boris Zbarsky wrote:
>>
>>> On 10/18/13 2:10 PM, Alex Keybl wrote:
>>>
>>>> The main purpose of Aurora as a channel is to get early feedback on
>>>> new Firefox versions with users that better represent our Beta/Release
>>>> populations.
>>>>
>>>
>>> A major purpose of Aurora (that I think we've totally failed to
>>> effectively publicize) is to provide a less-scary-than-nightly option
>>> for web developers who want to experiment with new functionality that's
>>> not enabled by default yet and provide feedback on it.  This is why
>>> various experimental stuff is enabled on Aurora.
>>>
>>
>> Periodic reminder that as long as we don't have one-checkbox
>> side-by-side installation of multiple channels (simultaneously runnable,
>> in separate profiles, sync hooked up automatically), web developers
>> cannot be bothered even with Beta, let alone Aurora.
>>
>
> This is a point we keep coming back to. If there aren't enough users on
> Aurora to be useful in terms of testing or bug discovery I would love to be
> able to use it to target web developers specifically ( we already recommend
> it ) along with some developer-centric features. This could be as simple as:
>

I think one change that would really help (suggested by choffman I believe)
is to rename Aurora to something that implies that it is a developer or
early release.

Stormy


>
> 1) prompt the user on first-run of Aurora if they want a 'developer'
> profile.
>
> 2) afterwards, only launch Aurora with the Aurora profile.
>
> I like the automagic sync idea only to a point - developers are going to
> tend to have different configurations / extensions / etc for a dev-centric
> browser.
>
> $0.02, Jeff
>
> ______________________________**_________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/**listinfo/dev-planning<https://lists.mozilla.org/listinfo/dev-planning>
>
Re: Coupled Train Model Francesco Lodolo [:flod] 10/22/13 8:14 AM
2013/10/22 Ben Francis <bfra...@mozilla.com>

> Are there any real blockers preventing us from doing this?


At what point do you plan to freeze strings, and let localizers translate
them?

Francesco
Re: Coupled Train Model Alexander Keybl 10/22/13 9:05 AM
We definitely plan to keep Firefox IS on the same heartbeat as we originally implemented, it's just a matter of how we align.

FxOS should not prevent us from making positive change on desktop, although it's of course an important consideration.

-Alex
_______________________________________________
dev-planning mailing list
dev-pl...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-planning

Re: Coupled Train Model Alex Keybl 10/22/13 9:08 AM
It's an extra 1-2 weeks of stabilization time, which will go a long way in stamping out amorphous beta-only issues.

We don't expect there to be an impact to Beta usage, given current quality of the Aurora channel. We do a good job of keeping Nightly in a usable state for the large majority of use cases. We have a discovery issue which we'd like to mitigate through more time with our largest pre-release population.

-Alex

----- Original Message -----
From: Karl Tomlinson <moz...@karlt.net>
To: dev-pl...@lists.mozilla.org
Sent: Mon, 21 Oct 2013 15:10:09 -0700 (PDT)
Subject: Re: Coupled Train Model

On Fri, 18 Oct 2013 14:10:52 -0400, Alex Keybl wrote:

> Aurora is a fantastic tool, but it's my team's opinion that it
> doesn't make great use of the full six weeks allotted to it in
> each development cycle. We find most major Aurora issues in the
> first week or so after the merge, and critical Beta-blocking
> issues are typically resolved within days thereafter. We've
> verified this premise by taking a look at bugs tracked for
> Firefox 24, fixed in the Aurora timeframe.
>
> We have an informal saying in Release Management that we should
> always "get new code in front of the most users as soon as
> possible". Given that goal, I'd like us to spend even more of
> the 18 weeks with the much larger Beta population, since that
> population is the most applicable to Release in diversity and
> quality on Aurora is already fairly high.

The benefits that I see from the proposal are:

1. Increase number of weeks of beta testing from 6 to hopefully 7.

2. Reduce time between m-c landing and release from 12-18 down to
 9-18. Assuming evenly distributed landings the average
 reduction is 1.5 weeks.

The main risk/disadvantage is lowering the quality of aurora and
beta. The Aurora channel would be almost as risky as Nightly at
merge time. Beta would have 2 weeks of stabilization instead of 6.

My experience is that there are plenty of bugs on Nightly that
take more than 2 weeks to filter through triage, get a fix and
review.

Those fixes, and others that are resolved before 2 weeks,
currently have some time of testing before they need to appear on
Beta. The new proposal either doesn't have testing time for these
fixes, or they don't make it in time for Beta users.

I would like to ask:

A. If most beta blocking issues are resolved within days, then how
 valuable is an extra week of beta testing?

B. Is it worth the cost of a lower quality beta?
 A lower quality beta would mean fewer users on that train, and
 therefore a narrower, less diverse, sample of testers and lower
 quality testing, IMHO more likely leading to a lower quality
 release product.
Re: Coupled Train Model Alex Keybl 10/22/13 9:16 AM
The main problem I see here is regularly secting a Nightly build that we can confidently push to Aurora users. May be too much churn for that population over the course of a cycle (as opposed to once at the beginning of the cycle).

It's obviously not a bad idea (it works for our competition) but we need to make sure our final solution fits our needs.

-Alex

----- Original Message -----
From: Eric Rescorla <e...@rtfm.com>
To: Lawrence Mandel <lma...@mozilla.com>
Cc: Chris Peterson <cpet...@mozilla.com>, dev-pl...@lists.mozilla.org
Sent: Mon, 21 Oct 2013 21:42:35 -0700 (PDT)
Subject: Re: Coupled Train Model

Apologies if I have missed this, but I've been reading this thread and I
find myself wondering why we don't adopt a model more like Chrome's
(http://www.chromium.org/getting-involved/dev-channel#TOC-Channels)

- 6 weeks on Nightly ("Canary")
- 6 weeks on Beta
- 6 weeks on Stable

Chrome also has a "Dev" channel which is just the latest Nightly which
has received some testing. We could just rebrand Aurora in that model
and move forward. If we seriously think that Aurora isn't getting much
testing, anyway, wouldn't this be simplest?

Again, apologies if this has been discussed and rejected. If such a
discussion exists, I will gladly accept a pointer.

Thanks,
-Ekr





On Mon, Oct 21, 2013 at 4:30 PM, Lawrence Mandel <lma...@mozilla.com>wrote:

> ----- Original Message -----
> > On 10/21/13 3:10 PM, Karl Tomlinson wrote:
> > > 1. Increase number of weeks of beta testing from 6 to hopefully 7.
> > >
> > > 2. Reduce time between m-c landing and release from 12-18 down to
> > > 9-18. Assuming evenly distributed landings the average
> > > reduction is 1.5 weeks.
> >
> > Plus, the estimated 7 weeks of Beta assumes that, because we currently
> > stablize 6 weeks of Nightly development with 2 weeks on Aurora, we can
> > stablize 9 weeks of Nightly development with 2 weeks on Aurora.
> >
> > Do we have reason to believe that Aurora stablization won't also
> > increase by 1.5x to 3 weeks? That would give us only 6 weeks of Beta
> > testing, i.e. no change from today's Beta duration, except with 3 weeks
> > of Aurora "bake time" (instead of 6) of 1.5x more code.
>
> This is an interesting thought. IIRC, the original proposal saw Nightly
> remain at 6 weeks, Aurora drop to 3 weeks and beta move to 9 weeks. With
> the 9/9 cycle we appear to be optimizing for development time. Do we
> need/want additional development time in each cycle? Does an additional
> week on beta really warrant the effort required to make this change?
>
> Lawrence
Re: Coupled Train Model Alex Keybl 10/22/13 9:21 AM
It puts a release in front of a much larger pre-release population for more of the cycle, allowing us to identify and fix issues farther from release.

The tweaked model also keeps developers focused in fewer versions, making sure recent changes are fresh in our memory.

It also allows for us to ship dot releases to our Beta users prior to release, a huge quality improvement from where we stand now.

It's not clear to me how spending less time with Aurora will impact QA bandwidth. It should instead help with focus (fewer Gecko versions to juggle).

-Alex

----- Original Message -----
From: anthony s hughes <anthony....@gmail.com>
To: dev-pl...@lists.mozilla.org
Sent: Mon, 21 Oct 2013 14:29:01 -0700 (PDT)
Subject: Re: Coupled Train Model

On Monday, October 21, 2013 10:54:44 AM UTC-7, Alex Keybl wrote:
> > I could be confused here, but I don't see how this is going to work with QA. The way I read the diagram, the two weeks of "fixing aurora" are going to happen at the same time as "doing stabilization point releases on beta for the new release". I don't think we want to force two conflicting priorities on the QA teams where they have to chose between supporting release stabilization and ensuring that aurora has stabilized to be uplifted into beta.
>
>
>
> I think that's a very narrow view of QA's bandwidth. Rarely do we have to choose between one investigation or another. We maintain 4 branches (3 pre-release), mind you. If we did have to choose, we'd of course prioritize Release though.

I cannot nor will I speak for the entire QA org; the following is purely my personal opinion. I'm not yet convinced that Aurora and the current train model are broken. I suspect it's how we are using them (or not) that is broken.

In my experience it's taken nearly everything Desktop QA has to keep up with the pace of uplifts and testing of Betas. I personally believe that QA should be living in Aurora but the current state of things does not afford us that opportunity. To get there we need better automation, tools for community involvement, and stricter, more clearly defined rules governing uplifts (especially on Beta).

I do not see how this proposal benefits the quality/stability of the product(s) we ship and I fear it might impose an even greater challenge on an already stretched QA team.
Re: Coupled Train Model Adam Roach 10/22/13 9:30 AM
On 10/22/13 09:34, Ben Francis wrote:
> On Tue, Oct 22, 2013 at 5:42 AM, Eric Rescorla <e...@rtfm.com> wrote:
>
>> I
>> find myself wondering why we don't adopt a model more like Chrome's
>> (http://www.chromium.org/getting-involved/dev-channel#TOC-Channels)
>>
>> - 6 weeks on Nightly ("Canary")
>> - 6 weeks on Beta
>> - 6 weeks on Stable
>
> Are there any real blockers preventing us from doing this? If we think
> Aurora isn't serving our needs well then removing it altogether in favour
> of the 6/6/6 model Chrome uses would certainly equal out a competitive
> disadvantage we currently have, with the time it takes to get a feature to
> market.

Right. Especially when it comes to new and developing technologies
(WebRTC comes to mind), being handicapped by six weeks contributes to an
overall impression that Firefox is simply less nimble than Chrome. On
top of new features taking longer, we have the additional impact that
any flaws that aren't major enough to uplift take a substantial chunk of
time to get into the hands of users.

I think that there's also an argument that we have a non-trivial
incremental cost in maintaining four release-train trees as compared to
three. So, even if we were to re-engineer the current
Nightly/Aurora/Beta/Release schedules to get from Nightly to Release in
12 weeks, we're still paying an additional price for producing a nearly
un-used product.

As a final point: any argument about why this schedule won't work needs
to adequately explain which of the factors that differentiate the
Firefox team different than the Chrome team are responsible for making
it okay for them but not for us. I'm not saying there aren't any such
reasons; just that I can't easily come up with any that are particularly
compelling.

I think this is a place where being bold would pay large dividends.

--
Adam Roach
Principal Platform Engineer
a...@mozilla.com
+1 650 903 0800 x863
Re: Coupled Train Model Ben Francis 10/22/13 9:59 AM
On Tue, Oct 22, 2013 at 4:14 PM, Francesco Lodolo [:flod]
<fl...@lodolo.net>wrote:

> 2013/10/22 Ben Francis <bfra...@mozilla.com>
>
> > Are there any real blockers preventing us from doing this?
>
> At what point do you plan to freeze strings, and let localizers translate
> them?


I don't have all the answers, I was just asking what the blockers are.
Thanks for bringing this one up, it was certainly something that was raised
at the session at the Summit in Toronto.

I think with Chromium they have a string freeze a couple of weeks before
moving a release to beta to provide a bit of a head start and then do the
bulk of the translations over the beta period, but it certainly reduces the
overall amount of time for localisations to happen. Perhaps we don't have
the level of resources needed to make this possible.

We could turn this problem into an opportunity with a campaign to recruit
volunteer translators from our beta testing community. "You are using beta
software, if you see untranslated strings and are able to translate them,
this is how you can help..." We'd certainly be taking a risk, but if we
successfully recruited more translators do you think we could speed up the
localisation process?
Re: Coupled Train Model Francesco Lodolo [:flod] 10/22/13 10:10 AM
Il 22/10/13 18:59, Ben Francis ha scritto:
> I think with Chromium they have a string freeze a couple of weeks
> before moving a release to beta to provide a bit of a head start and
> then do the bulk of the translations over the beta period, but it
> certainly reduces the overall amount of time for localisations to
> happen. Perhaps we don't have the level of resources needed to make
> this possible.
Which means that beta releases will be low-quality, half-translated
products for the first part of the cycle.

That's far from optimal: low quality in localization means perceived low
quality of the product itself (poor Firefox in Italian = poor Firefox,
period), and as a localizer I will be less than happy to have something
like this available to the public (it would make my work look sloppy).
> We could turn this problem into an opportunity with a campaign to
> recruit volunteer translators from our beta testing community. "You
> are using beta software, if you see untranslated strings and are able
> to translate them, this is how you can help..." We'd certainly be
> taking a risk, but if we successfully recruited more translators do
> you think we could speed up the localisation process?
I personally think that this would just lower the quality of the product
we ship. There are team leaders and communities in place to ensure that
we ship the best product possible (quality obviously varies from locale
to locale), introducing occasional localizations/localizers would make
that almost impossible.

Francesco

Re: Coupled Train Model Daniel Veditz 10/22/13 10:19 AM
On 10/21/2013 10:46 PM, Asa Dotzler wrote:
> Was there any discussion at the Summit session about how the release
> cadence maps not only to the efficient use of limited contributor
> resources, but also how it impacts our end users?

> My personal opinion is that our end users would love Firefox at least a
> little bit more, possibly a lot more, if our release cadence was
> quarterly.  But that's a very different discussion and perhaps one that
> was already ruled out at the Summit sessions?

we did talk about it. The proposal lengthens the time between
feature-changing updates which everyone (most?) agreed would be welcomed
by users who generally don't like change and interruption. My personal
concern is that we then might need to inject a mid-term security update
on a regular basis which doesn't help as much, although such an update
would at least not change any user-facing features or break add-ons.

-Dan Veditz
Re: Coupled Train Model Ben Francis 10/22/13 11:07 AM
On Tue, Oct 22, 2013 at 6:10 PM, Francesco Lodolo [:flod]
<fl...@lodolo.net>wrote:

> Il 22/10/13 18:59, Ben Francis ha scritto:
>
>  I think with Chromium they have a string freeze a couple of weeks before
>> moving a release to beta to provide a bit of a head start and then do the
>> bulk of the translations over the beta period, but it certainly reduces the
>> overall amount of time for localisations to happen. Perhaps we don't have
>> the level of resources needed to make this possible.
>>
> Which means that beta releases will be low-quality, half-translated
> products for the first part of the cycle.
>

Of course, and that would presumably be intentional. By combining Aurora
and Beta into one channel we would have the cost of lower quality at the
start of the Beta cycle, but the benefit of more eyes finding bugs with the
intention of shortening the stabilisation period. That's not necessarily
the right thing for Mozilla, but it is what one of our competitors does. As
Adam points out, if there's a reason we can't move as fast as they do, we
should identify the reasons for that.

The argument I believe is being put forward by Release Management is that
we don't have enough users using Aurora builds to see any benefit from
them. We can't simulatenously have the goal of putting lower quality
software in front of more users and putting lower quality software in front
of less users. Beta users have opted in to getting new features sooner in
return for a less polished product, we'd just be tweaking those dials a
little. That's not necessarily a bad thing if we communicate it well.

That's far from optimal: low quality in localization means perceived low
> quality of the product itself (poor Firefox in Italian = poor Firefox,
> period), and as a localizer I will be less than happy to have something
> like this available to the public (it would make my work look sloppy).


Aurora is available to the public too, it's just that less people use it.
That's been identified as a problem that needs solving, if that is a
problem then this is one potential solution.

Ben
Re: Coupled Train Model Matt Brubeck 10/22/13 11:13 AM
On 10/21/2013 4:01 PM, Chris Peterson wrote:
> Plus, the estimated 7 weeks of Beta assumes that, because we currently
> stablize 6 weeks of Nightly development with 2 weeks on Aurora, we can
> stablize 9 weeks of Nightly development with 2 weeks on Aurora.

I don't *think* this would be a very big problem.  New bugs and
regressions that land early in the Nightly cycle can be fixed on
Nightly, before ever reaching Aurora.  I don't expect there are a large
number of bugs that are critical enough to fix in Aurora, and yet would
be allowed remain unfixed on Nightly for 7 or 9 weeks.
Re: Coupled Train Model chris hofmann 10/22/13 11:37 AM
On 10/22/13 11:07 AM, Ben Francis wrote:
>
>
> Aurora is available to the public too, it's just that less people use it.
> That's been identified as a problem that needs solving, if that is a
> problem then this is one potential solution.

there are many possible solutions.

aurora's never gotten the attention publicity and understanding that
would allow it to grow.

It's impossible to find navigating from the mozilla.org home page.

People know that a beta is, but what is an "aurora?"   We made up that
name, but it turns out that was probably a failed attempt that was never
corrected.

If I happen to know that "aurora" actually existed, and what it was
about, I might google search for it and find
http://www.mozilla.org/en-US/firefox/aurora/

Then on that page I'd find a section on that page that says "Read about
the latest Aurora features" and see some articles that are over a year
and a half old like:

#


        The latest Firefox Aurora is now available for download and
        testing!
        <http://blog.mozilla.com/futurereleases/2012/03/19/the-latest-firefox-aurora-is-now-available-for-download-and-testing/>

March 19, 2012 . Firefox Aurora

Again,  if we are really interested in growing aurora we should be
defining what its about and then growing the story and spreading the
world.   There is a great missed opportunity to promote aurora as *the*
mozilla web developer release; changing its name so its easy to find and
understand, and start really finding out how many web developers find
and use a mozilla browser that's really targeted at them and useful to them.

-chofmann

>
> Ben
Re: Coupled Train Model Benjamin Smedberg 10/22/13 11:49 AM
On 10/22/2013 2:07 PM, Ben Francis wrote:
> Of course, and that would presumably be intentional. By combining Aurora
> and Beta into one channel we would have the cost of lower quality at the
> start of the Beta cycle, but the benefit of more eyes finding bugs with the
> intention of shortening the stabilisation period.
Your argument assumes the beta population as a given. My experience has
been that our beta population uses addons in ways that are fairly
similar to our release population, and expect a fairly high quality bar.
If we trade off quality early in the cycle, and especially if addons
don't work properly, I expect that we'd see a substantial dropoff in the
number of beta users. This also is likely to skew the beta population
away from release in ways that make it difficult to measure and be
confident in various telemetry/crash numbers.

My primary worry about this proposal is that the two-or-so weeks of beta
are going to cause more addons to be incompatible at the first beta,
which is going to annoy and drive away the beta user base.

> Aurora is available to the public too, it's just that less people use
> it. That's been identified as a problem that needs solving

It has certainly been identified as a fact of the current situation.
Aurora is not attractive to casual users because of the addon situation,
and it's not attractive to most hardcore users because Nightly gets you
cool things faster. But I'm not sure that there is agreement that it's a
problem we should solve.

--BDS

Re: Coupled Train Model Axel Hecht 10/22/13 11:59 AM
I don't know anything about the l10n ecosystem of chrome, and I'd rather
not speculate. I didn't find any current docs, either. So if someone
happens to know someone that knows someone that knows... :-)

Caveats:

One of the major metrics of quality of localization is consistency, so
"the more the merrier" doesn't hold for l10n.

Also, taking contributions is a tough task, if you're having 100 new
strings, and you need to do 100 patch reviews, it's not all that much
help to the person that needs to do the reviews.

But to reiterate the ask that chofmann and I raised:

We need detailed tasks and expectations for things, and then we can put
them on a calendar.

Axel
Re: Coupled Train Model Karl Tomlinson 10/22/13 12:07 PM
Daniel Veditz writes:

> On 10/21/2013 10:46 PM, Asa Dotzler wrote:
>> My personal opinion is that our end users would love Firefox at least a
>> little bit more, possibly a lot more, if our release cadence was
>> quarterly.  But that's a very different discussion and perhaps one that
>> was already ruled out at the Summit sessions?
>
> The proposal lengthens the time between
> feature-changing updates which everyone (most?) agreed would be
> welcomed by users who generally don't like change and
> interruption. My personal concern is that we then might need to
> inject a mid-term security update on a regular basis which doesn't
> help as much, although such an update would at least not change
> any user-facing features or break add-ons.

Although there would be fewer releases having user-facing changes,
there would still be the same rate of using facing changes.
The difference with the longer period between release would be
that more changes would be in each update.

Having more frequent releases provides that opportunity to change
things gradually.
Re: Coupled Train Model Karl Tomlinson 10/22/13 12:11 PM
Alexander Keybl writes:

> It also allows for us to ship dot releases to our Beta users
> prior to release, a huge quality improvement from where we stand
> now.

I wonder what the proportion of fixes in dot releases are security sensitive.

Perhaps you've already thought about this, but with security
sensitive fixes, we may not want to make the change public any
longer than necessary before shipping the fix.
Re: Coupled Train Model Ben Francis 10/22/13 12:27 PM
On Fri, Oct 18, 2013 at 7:10 PM, Alex Keybl <ake...@mozilla.com> wrote:

> A chart explaining the Coupled Train Model can be found at
> https://pbs.twimg.com/media/BV7r23WCMAAROMP.png:large
>

Another way at looking at this from the users' point of view might be that
there are three release channels.

Excuse my dodgy diagram
http://people.mozilla.org/~bfrancis/release_channels.png but:

* dev - nightly builds, the bleeding edge.
* testing - alpha or beta builds about once a week for testing new features
before they are stable.
* stable - a stable release every six weeks, then security point releases
if necessary.

The diagram shows six week cycles (as an example), where the first two
releases on the testing channel are alpha and the following four are beta.
That would allow a user on the testing channel to opt out of alpha releases
if they'd prefer a little bit more stability - potentially solving the
addon incompatibility problem Benjamin mentioned.

Of course all of this is built on the hypothesis that we could safely
change the ratio between new feature development and stabilisation from 1:2
to 1:1, which seems a more important decision than the length of the cycle.

Ben
Re: Coupled Train Model Anthony Hughes 10/22/13 12:45 PM
----- Original Message -----
> From: "Alexander Keybl" <ake...@mozilla.com>
> To: "anthony s hughes" <anthony....@gmail.com>
> Cc: dev-pl...@lists.mozilla.org
> Sent: Tuesday, October 22, 2013 9:21:16 AM
> Subject: Re: Coupled Train Model
>
> It puts a release in front of a much larger pre-release population for more
> of the cycle, allowing us to identify and fix issues farther from release.
>
> The tweaked model also keeps developers focused in fewer versions, making
> sure recent changes are fresh in our memory.
>
> It also allows for us to ship dot releases to our Beta users prior to
> release, a huge quality improvement from where we stand now.

I can certainly understand that point of view in theory but I'm a bit sceptical it will work out that way in practice; at least not without far more stricter and more broadly understood rules about what gets uplift approval.

> It's not clear to me how spending less time with Aurora will impact QA
> bandwidth. It should instead help with focus (fewer Gecko versions to
> juggle).

Then why not a 6 week Nightly and 12 week Beta cycle?
Re: Coupled Train Model Adam Roach 10/22/13 12:55 PM
On 10/22/13 14:45, Anthony Hughes wrote:
> Then why not a 6 week Nightly and 12 week Beta cycle?

See my earlier comments about agility versus Chrome.
Re: Coupled Train Model Chris Peterson 10/22/13 1:02 PM
On 10/22/13 9:21 AM, Alexander Keybl wrote:
> It puts a release in front of a much larger pre-release population for more of the cycle, allowing us to identify and fix issues farther from release.

AFAICT, the Beta test time would only increase from 6 weeks to 7 because
the first 2 weeks of the 9 week cycle are spent on Aurora. If 2 weeks on
Aurora is little better than 6, does Beta have a different bug
distribution that would make 7 weeks more useful than 6?

Advantages of the coupled train model:
* Beta test time increases from 6 weeks to 7
* A feature's landing-to-release time decreases from 12-18 weeks to 9-18.
* Fewer Gecko versions to maintain (because Aurora and Beta have the
same Gecko version)

Disadvantages:
* Beta users get software with 4 weeks less Aurora bake time
* L10n has less time to localize features
* Addon developers have less time to fix their addons
* (Even more) confusing marketing story around Aurora
* People won't have much reason to run Aurora after the first 2 weeks

Am I missing any? I don't think these advantages outweigh these
disadvantages.

Traditional software development has a long code freeze period of just
fixing bugs. In a sense, we have a 12 week development cycle and Aurora
is just our code freeze time (but run in parallel).


cpeterson
Re: Coupled Train Model Alex Keybl 10/22/13 2:24 PM
> AFAICT, the Beta test time would only increase from 6 weeks to 7 because the first 2 weeks of the 9 week cycle are spent on Aurora. If 2 weeks on Aurora is little better than 6, does Beta have a different bug distribution that would make 7 weeks more useful than 6?


We expect to spend 1-2 weeks with the Aurora population depending on the criticality of issues identified.

-Alex
Re: Coupled Train Model jorgev 10/22/13 3:33 PM
I don't think the new model will make our add-on situation any
different. Add-on developers and add-on users test their add-ons on one
or more of the channels, and that will continue to happen. An add-on
that regularly breaks on beta will continue to regularly break on beta.
An add-on that is kept up to date with Aurora will continue to be up to
date with Aurora. What matters is which channel the developer is
monitoring and how many of the add-on's users are on each channel in
order to nag the developer about any problems.

Our compatibility reports are generally very late and only surface
during the beta cycle. We're trying to catch up with that and inform
developers about incompatibilities during the Aurora cycle, but that's
besides the point. What's true is that right now we're not communicating
these changes until a few weeks into beta, so I don't think the proposed
changes will make a difference in that process or how add-on developers
respond to it.

Jorge
Re: Coupled Train Model jorgev 10/22/13 3:33 PM
On 10/22/13 12:49 PM, Benjamin Smedberg wrote:
Re: Coupled Train Model jorgev 10/22/13 3:42 PM
On 10/22/13 2:02 PM, Chris Peterson wrote:
> On 10/22/13 9:21 AM, Alexander Keybl wrote:
>> It puts a release in front of a much larger pre-release population for
>> more of the cycle, allowing us to identify and fix issues farther from
>> release.
>
> AFAICT, the Beta test time would only increase from 6 weeks to 7 because
> the first 2 weeks of the 9 week cycle are spent on Aurora. If 2 weeks on
> Aurora is little better than 6, does Beta have a different bug
> distribution that would make 7 weeks more useful than 6?
>
> Advantages of the coupled train model:
> * Beta test time increases from 6 weeks to 7
> * A feature's landing-to-release time decreases from 12-18 weeks to 9-18.
> * Fewer Gecko versions to maintain (because Aurora and Beta have the
> same Gecko version)
>
> Disadvantages:
> * Beta users get software with 4 weeks less Aurora bake time
> * L10n has less time to localize features
> * Addon developers have less time to fix their addons

Add-on developers will most likely welcome the longer cycles. Our
experience is that most of them test their add-ons on beta or even
release, even if we recommend Aurora.

Jorge
Re: Coupled Train Model Lawrence Mandel 10/22/13 4:49 PM
----- Original Message -----
> The main problem I see here is regularly secting a Nightly build that we can
> confidently push to Aurora users. May be too much churn for that population
> over the course of a cycle (as opposed to once at the beginning of the
> cycle).

How about moving to the weekly integration build model. In this model one build a week (say, on Thursday) is selected for smoke testing. If the smoke tests pass, the build is promoted to stable. In this model Aurora becomes the stable branch.

Lawrence

>
> It's obviously not a bad idea (it works for our competition) but we need to
> make sure our final solution fits our needs.
>
> -Alex
>
> ----- Original Message -----
> From: Eric Rescorla <e...@rtfm.com>
> To: Lawrence Mandel <lma...@mozilla.com>
> Cc: Chris Peterson <cpet...@mozilla.com>, dev-pl...@lists.mozilla.org
> Sent: Mon, 21 Oct 2013 21:42:35 -0700 (PDT)
> Subject: Re: Coupled Train Model
>
> Apologies if I have missed this, but I've been reading this thread and I
> find myself wondering why we don't adopt a model more like Chrome's
> (http://www.chromium.org/getting-involved/dev-channel#TOC-Channels)
>
> - 6 weeks on Nightly ("Canary")
> - 6 weeks on Beta
> - 6 weeks on Stable
>
> Chrome also has a "Dev" channel which is just the latest Nightly which
> has received some testing. We could just rebrand Aurora in that model
> and move forward. If we seriously think that Aurora isn't getting much
> testing, anyway, wouldn't this be simplest?
>
> Again, apologies if this has been discussed and rejected. If such a
> discussion exists, I will gladly accept a pointer.
>
> Thanks,
> -Ekr
>
>
>
>
>
> On Mon, Oct 21, 2013 at 4:30 PM, Lawrence Mandel <lma...@mozilla.com>wrote:
>
> > ----- Original Message -----
> > > On 10/21/13 3:10 PM, Karl Tomlinson wrote:
> > > > 1. Increase number of weeks of beta testing from 6 to hopefully 7.
> > > >
> > > > 2. Reduce time between m-c landing and release from 12-18 down to
> > > > 9-18. Assuming evenly distributed landings the average
> > > > reduction is 1.5 weeks.
> > >
> > > Plus, the estimated 7 weeks of Beta assumes that, because we currently
> > > stablize 6 weeks of Nightly development with 2 weeks on Aurora, we can
> > > stablize 9 weeks of Nightly development with 2 weeks on Aurora.
> > >
> > > Do we have reason to believe that Aurora stablization won't also
> > > increase by 1.5x to 3 weeks? That would give us only 6 weeks of Beta
> > > testing, i.e. no change from today's Beta duration, except with 3 weeks
> > > of Aurora "bake time" (instead of 6) of 1.5x more code.
> >
> > This is an interesting thought. IIRC, the original proposal saw Nightly
> > remain at 6 weeks, Aurora drop to 3 weeks and beta move to 9 weeks. With
> > the 9/9 cycle we appear to be optimizing for development time. Do we
> > need/want additional development time in each cycle? Does an additional
> > week on beta really warrant the effort required to make this change?
> >
> > Lawrence
Re: Coupled Train Model Eric Rescorla 10/22/13 4:58 PM
I like this a lot.

-Ekr



On Tue, Oct 22, 2013 at 4:49 PM, Lawrence Mandel <lma...@mozilla.com>wrote:
Re: Coupled Train Model Chris Peterson 10/22/13 5:18 PM
On 10/22/13, 4:49 PM, Lawrence Mandel wrote:
> ----- Original Message -----
>> >The main problem I see here is regularly secting a Nightly build that we can
>> >confidently push to Aurora users. May be too much churn for that population
>> >over the course of a cycle (as opposed to once at the beginning of the
>> >cycle).
>
> How about moving to the weekly integration build model. In this model one build a week (say, on Thursday) is selected for smoke testing. If the smoke tests pass, the build is promoted to stable. In this model Aurora becomes the stable branch.

Aurora users are willing to install pre-beta software and receive daily
updates, so I don't think churn is a big concern. In fact, if we update
Aurora users just once or twice a week, we might attract some Beta users
that were turned off by Aurora's daily updates.

An analogy:

mozilla-inbound : mozilla-cental :: Nightly : Aurora


cpeterson
Re: Coupled Train Model Karl Tomlinson 10/22/13 5:49 PM
Jorge Villalobos writes:

> On 10/22/13 12:49 PM, Benjamin Smedberg wrote:
>> Your argument assumes the beta population as a given. My experience has
>> been that our beta population uses addons in ways that are fairly
>> similar to our release population, and expect a fairly high quality bar.
>> If we trade off quality early in the cycle, and especially if addons
>> don't work properly, I expect that we'd see a substantial dropoff in the
>> number of beta users. This also is likely to skew the beta population
>> away from release in ways that make it difficult to measure and be
>> confident in various telemetry/crash numbers.
>>
>> My primary worry about this proposal is that the two-or-so weeks of beta
>> are going to cause more addons to be incompatible at the first beta,
>> which is going to annoy and drive away the beta user base.

> What matters is which channel the developer is
> monitoring and how many of the add-on's users are on each channel in
> order to nag the developer about any problems.

Hopefully anyone on a channel downstream from what the developer is
watching will not have problems (too often).

> Our compatibility reports are generally very late and only surface
> during the beta cycle. We're trying to catch up with that and inform
> developers about incompatibilities during the Aurora cycle, but that's
> besides the point. What's true is that right now we're not communicating
> these changes until a few weeks into beta, so I don't think the proposed
> changes will make a difference in that process or how add-on developers
> respond to it.

If we're having trouble catching up with that, then it sounds like
there is a risk that these reports will arrive even later in the
beta cycle.
Re: Coupled Train Model Lawrence Mandel 10/22/13 6:11 PM
Right. My suggestion was based on the idea that the 6 weeks on Aurora are not very useful as a time between Nightly and Beta but the more stable Aurora channel is useful to certain groups like QA and l10n and (ideally) Web devs. As such, let's give these groups a stable channel that runs concurrently to Nightly while cutting 6 weeks from the cycle. Note that I haven't discussed this idea with anyone from QA or l10n so I don't know if this will be useful to them. (I expect that l10n will need some sort of string freeze toward the end of the Nightly cycle to make this useful which may or may not be a show stopper.)

Lawrence
Re: Coupled Train Model Gervase Markham 10/23/13 4:39 AM
On 22/10/13 16:00, Stormy Peters wrote:
> I think one change that would really help (suggested by choffman I believe)
> is to rename Aurora to something that implies that it is a developer or
> early release.

Er... Alpha? :-)

Gerv

Re: Coupled Train Model Eric Rescorla 10/23/13 5:03 AM
We could be really unimaginative and call it "Dev"

-Ekr
Re: Coupled Train Model Rob Campbell 10/23/13 6:02 AM
"Minotaur".
Re: Coupled Train Model Alexander Keybl 10/23/13 6:07 AM
This is basically accomplished through frequent merges to Aurora instead of only using Aurora as a step at the beginning of the cycle (nbp suggested something similar). I specify merges because I think promotion of specific builds as opposed to maintaining a separate branch may prevent us from meeting Aurora's quality bar, especially when critical issues go uncaught by smoketesting. I suspect the Aurora to Beta merge will still be delayed from merge day, so I see this as an addition to the Coupled Train Model.

Frequency of merges would be based upon feedback from QA on their bandwidth for qualifying a new merge. Final call will be with my team, once we've taken feedback from all involved.

Do people think that we should continue to explore lengthening the cycle if we spend 1:1 development:convergence, even given this modified proposal?

-Alex

----- Original Message -----
From: Lawrence Mandel <lma...@mozilla.com>
To: Alexander Keybl <ake...@mozilla.com>
Cc: Eric Rescorla <e...@rtfm.com>, Chris Peterson <cpet...@mozilla.com>, dev-pl...@lists.mozilla.org
Sent: Tue, 22 Oct 2013 16:49:49 -0700 (PDT)
Subject: Re: Coupled Train Model

----- Original Message -----
> The main problem I see here is regularly secting a Nightly build that we can
> confidently push to Aurora users. May be too much churn for that population
> over the course of a cycle (as opposed to once at the beginning of the
> cycle).

How about moving to the weekly integration build model. In this model one build a week (say, on Thursday) is selected for smoke testing. If the smoke tests pass, the build is promoted to stable. In this model Aurora becomes the stable branch.

Lawrence

>
> It's obviously not a bad idea (it works for our competition) but we need to
> make sure our final solution fits our needs.
>
> -Alex
>
> ----- Original Message -----
> From: Eric Rescorla <e...@rtfm.com>
> To: Lawrence Mandel <lma...@mozilla.com>
> Cc: Chris Peterson <cpet...@mozilla.com>, dev-pl...@lists.mozilla.org
> Sent: Mon, 21 Oct 2013 21:42:35 -0700 (PDT)
> Subject: Re: Coupled Train Model
>
> Apologies if I have missed this, but I've been reading this thread and I
> find myself wondering why we don't adopt a model more like Chrome's
> (http://www.chromium.org/getting-involved/dev-channel#TOC-Channels)
>
> - 6 weeks on Nightly ("Canary")
> - 6 weeks on Beta
> - 6 weeks on Stable
>
> Chrome also has a "Dev" channel which is just the latest Nightly which
> has received some testing. We could just rebrand Aurora in that model
> and move forward. If we seriously think that Aurora isn't getting much
> testing, anyway, wouldn't this be simplest?
>
> Again, apologies if this has been discussed and rejected. If such a
> discussion exists, I will gladly accept a pointer.
>
> Thanks,
> -Ekr
>
>
>
>
>
> On Mon, Oct 21, 2013 at 4:30 PM, Lawrence Mandel <lma...@mozilla.com>wrote:
>
> > ----- Original Message -----
> > > On 10/21/13 3:10 PM, Karl Tomlinson wrote:
> > > > 1. Increase number of weeks of beta testing from 6 to hopefully 7.
> > > >
> > > > 2. Reduce time between m-c landing and release from 12-18 down to
> > > > 9-18. Assuming evenly distributed landings the average
> > > > reduction is 1.5 weeks.
> > >
> > > Plus, the estimated 7 weeks of Beta assumes that, because we currently
> > > stablize 6 weeks of Nightly development with 2 weeks on Aurora, we can
> > > stablize 9 weeks of Nightly development with 2 weeks on Aurora.
> > >
> > > Do we have reason to believe that Aurora stablization won't also
> > > increase by 1.5x to 3 weeks? That would give us only 6 weeks of Beta
> > > testing, i.e. no change from today's Beta duration, except with 3 weeks
> > > of Aurora "bake time" (instead of 6) of 1.5x more code.
> >
> > This is an interesting thought. IIRC, the original proposal saw Nightly
> > remain at 6 weeks, Aurora drop to 3 weeks and beta move to 9 weeks. With
> > the 9/9 cycle we appear to be optimizing for development time. Do we
> > need/want additional development time in each cycle? Does an additional
> > week on beta really warrant the effort required to make this change?
> >
> > Lawrence
Re: Coupled Train Model Ted Mielczarek 10/23/13 7:07 AM
I think we explicitly rejected Alpha because of the negative connotation
towards the quality of the software. The Aurora channel may be pre-beta,
but it's nothing like what testers would traditionally call
alpha-quality software.

-Ted

Re: Coupled Train Model Axel Hecht 10/23/13 7:14 AM
Persona ... just kidding.

Axel
Re: Coupled Train Model Axel Hecht 10/23/13 7:16 AM
I don't see room for l10n here?

Axel
Re: Coupled Train Model Lawrence Mandel 10/23/13 10:43 AM
----- Original Message -----
> I don't see room for l10n here?

In a model like this HEAD/Nightly is always open. The merges (as Alex phrased it) happen on a specific release schedule. I think Alex is right about maintaining a separate Aurora branch. We need to work a feature freeze into this model that allows suitable time for l10n to do their thing.

Lawrence
Re: Coupled Train Model jorgev 10/23/13 1:25 PM
Depends on what you mean by "later". If we keep the current pace, we
will still publish the blog post ~3 weeks before release and the
compatibility bump ~1 week before release. In the new model that would
mean more weeks into beta, so that could be called "later".

Either way, we need to catch up and cut down at least a couple of weeks
in this process. It's really just a matter of doing things earlier, but
there's almost always something else in the way.

Jorge
Re: Coupled Train Model Jeff Griffiths 10/23/13 2:32 PM


On 10/22/2013, 8:00 AM, Stormy Peters wrote:
...
> I think one change that would really help (suggested by choffman I
> believe) is to rename Aurora to something that implies that it is a
> developer or early release.
>
> Stormy

Huh. I hadn't really considered this because I actually like Aurora (
especially coupled with the visual of the logo ).

Assuming we don't call it Persona, nothing springs to mind.
Re: Coupled Train Model Matt Basta 10/23/13 3:07 PM
The name Aurora is the single biggest factor that contributes to it not having many users. I've never met a single non-Mozillian that had any idea what Aurora was or what it was supposed to represent. It may be a cute name and have cute branding but if users don't know what the hell it is, then we've failed. Let's call it something more generic, like Alpha or Early or something like that.

----- Original Message -----
From: "Jeff Griffiths" <jgrif...@mozilla.com>
To: "Stormy Peters" <sto...@mozilla.com>
Cc: dev-pl...@lists.mozilla.org, "Zack Weinberg" <za...@panix.com>
Sent: Wednesday, October 23, 2013 2:32:02 PM
Subject: Re: Coupled Train Model



Re: Coupled Train Model Zack Weinberg 10/23/13 4:58 PM
On 2013-10-22 11:00 AM, Stormy Peters wrote:
> On Mon, Oct 21, 2013 at 2:24 PM, Jeff Griffiths <jgrif...@mozilla.com>wrote:
>> On 10/18/2013, 2:25 PM, Zack Weinberg wrote:
>>> Periodic reminder that as long as we don't have one-checkbox
>>> side-by-side installation of multiple channels (simultaneously runnable,
>>> in separate profiles, sync hooked up automatically), web developers
>>> cannot be bothered even with Beta, let alone Aurora.
>>
>> This is a point we keep coming back to. If there aren't enough users on
>> Aurora to be useful in terms of testing or bug discovery I would love to be
>> able to use it to target web developers specifically ( we already recommend
>> it ) along with some developer-centric features. This could be as simple as:
>
> I think one change that would really help (suggested by choffman I believe)
> is to rename Aurora to something that implies that it is a developer or
> early release.

I'm not opposed to a name change, but I don't think it's nearly as
important as the functionality of being able to install multiple
channels *and run them simultaneously*.

It's easy to tell people that if they want to play around with
upcomingness they can get either Beta, Aurora, or even Nightly builds.
But not being able to run Release at the same time is a deal breaker,
because more than anything else, webdevs care about matching the setup
_their_ users have.

The last time I tried it, installing Beta (with the default installer
settings) *overwrote* the Release install on the same machine, requiring
a complete uninstall and reinstall to recover.  You _can_ persuade the
installer to not do that, and with additional mucking around with the
profile manager and desktop shortcut command lines you can even get all
the way to what's desired.  There might even be accurate instructions on
MDN! (there weren't last time I looked, which admittedly was over a year
ago).

But it's sufficiently troublesome that I think it is unfair to ask of a
group of people who largely do not consider themselves part of the
"Mozilla community" and whom we are, as they may see it, asking to do us
a favor.

zw
Re: Coupled Train Model Mark Finkle 10/23/13 5:17 PM
Something I have been thinking about lately is how we could make non-trivial changes to the application after it hits Beta. Yes, Beta. Google does something like this for Chrome for Android. Many changes we see in Chrome for Android (Beta) never make it to the release version.

Firefox for Android sees an insignificant user base until it hits Beta. Once on Beta, we start getting a lot of valuable feedback. Mozilla's traditional viewpoint on Beta is "no significant changes" which can put Firefox for Android is a sucky situation.

I want to move to a process that gets a Beta sized audience using the product, but still allowing Mozilla to make non-trivial changes based on early feedback. I see the 9 week Beta cycle as a way to do that.

Finkle

----- Original Message -----

> The main purpose of Aurora as a channel is to get early feedback on new
> Firefox versions with users that better represent our Beta/Release
> populations. Aurora users knowingly accept a lower quality bar than
> Beta/Release, as well as more frequent updates.

> Aurora is a fantastic tool, but it's my team's opinion that it doesn't make
> great use of the full six weeks allotted to it in each development cycle. We
> find most major Aurora issues in the first week or so after the merge, and
> critical Beta-blocking issues are typically resolved within days thereafter.
> We've verified this premise by taking a look at bugs tracked for Firefox 24,
> fixed in the Aurora timeframe.

> We have an informal saying in Release Management that we should always "get
> new code in front of the most users as soon as possible". Given that goal,
> I'd like us to spend even more of the 18 weeks with the much larger Beta
> population, since that population is the most applicable to Release in
> diversity and quality on Aurora is already fairly high. Here's my proposed
> "Coupled Train Model" (thanks to curtisk for the naming):

> * change the desktop/android train model to have two 9-week cycles rather
> than three 6-week cycles (still 18 weeks start to finish)
> * enable Aurora updates the day after release, rather than at the end of the
> week
> * merge mozilla-aurora to mozilla-beta as soon as critical quality issues
> have been resolved on both Aurora and Release (1-2 weeks after that)
> ** we'd utilize the 1-2 week overlap to push dot releases to the Beta
> population prior to Release (bonus functionality!)
> * continue to have certain features only enabled on Aurora and lower
> * continue to use mozilla-aurora and the Aurora population as a method of
> testing fixes prior to uplift to mozilla-beta
> * allow developers to focus on two pre-release Gecko versions at a time,
> rather than three

> This proposal sees the Aurora population as a shorter, intermediate step
> towards Beta testing rather than an equally weighted phase of development
> (in terms of time). A chart explaining the Coupled Train Model can be found
> at https://pbs.twimg.com/media/BV7r23WCMAAROMP.png:large

> Things we're still considering, and we're meeting separately on:

> * a start date - current proposal is starting with Firefox 30 in the new year
> * a new string freeze date - current proposal is upon the merge to Beta,
> since features that aren't ready for release will be backed out by that
> point
> * an API freeze date, for add-on/plugin authors - still need to determine a
> hard date, but somewhere shortly after the merge to Beta
> * when (and from where) to branch future FxOS releases
> * security update frequency and the bar for a security dot release (given
> platform updates every 9 weeks)
> * how we would want to communicate the Aurora/Beta "releases" (perhaps
> together?)

> Thanks to all those who have already provided feedback (and early support) at
> the Open Sessions on Releases in Santa Clara and Toronto. I'd love to hear
> even more feedback from those community members who weren't able to attend.

> -Alex
Re: Coupled Train Model Gervase Markham 10/24/13 1:59 AM
Does alpha still really mean "crashes a lot; unusable for daily work"?
Stormy said we wanted a name which implies that it's a developer or
early release. If that name isn't "beta", then the step before beta is
"alpha".

Gerv

Re: Coupled Train Model Girish Sharma 10/24/13 2:20 AM
I think using the word "Dev" will be really good especially when Chrome
also uses the word Dev for their build of Chrome which is equivalent to
Aurora. This means that users who use Chrome will already be familiar with
the term "Dev" builds and its purpose.

Also, here at DevTools, we also focus on a train-ly basis post highlighting
new features in Aurora. This will be in line with the new name (if chosen)
"Dev".

In fact, there was a proposal of creating a new train, possible in parallel
with Aurora, named related to "Dev" which can be the representative build
for DevTools before the actual release build.
> _______________________________________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-planning
>



--
Girish Sharma
B.Tech(H), Civil Engineering,
Indian Institute of Technology, Kharagpur
Re: Coupled Train Model Axel Hecht 10/24/13 3:20 AM
On 10/24/13 12:07 AM, Matt Basta wrote:
> The name Aurora is the single biggest factor that contributes to it not having many users. I've never met a single non-Mozillian that had any idea what Aurora was or what it was supposed to represent. It may be a cute name and have cute branding but if users don't know what the hell it is, then we've failed. Let's call it something more generic, like Alpha or Early or something like that.
I think the name doesn't matter. We're hiding aurora as good as we
possibly can, and IIRC, very intentionally so. Early on, Grace did a
very targeted outreach, with talks and schwag and whatnot. And then
everything got pulled and hidden.

If you look at google results on what links to aurora pages, you find
promo material from Firefox 4, 5 and 6. And that's it, basically. Even
https://developer.mozilla.org/en-US/search?q=aurora shows mostly l10n pages.

We don't have users on Aurora because we do our best to keep them from
getting it.

Axel
Re: Coupled Train Model Philip Chee 10/24/13 4:47 AM
On 24/10/2013 06:07, Matt Basta wrote:
> The name Aurora is the single biggest factor that contributes to it
> not having many users. I've never met a single non-Mozillian that had
> any idea what Aurora was or what it was supposed to represent. It may
> be a cute name and have cute branding but if users don't know what
> the hell it is, then we've failed. Let's call it something more
> generic, like Alpha or Early or something like that.

I think Thunderbird has a "Early Bird" branch. So perhaps by analogy we
could use "Early Fox". What are baby foxes called anyway?

Phil

--
Philip Chee <phi...@aleytys.pc.my>, <phili...@gmail.com>
http://flashblock.mozdev.org/ http://xsidebar.mozdev.org
Guard us from the she-wolf and the wolf, and guard us from the thief,
oh Night, and so be good for us to pass.
Re: Coupled Train Model Philip Chee 10/24/13 4:50 AM
We could call it "Aleph" close enough to "Alpha" but without the
negative (if any) connotations.
Re: Coupled Train Model Chris Peterson 10/24/13 9:59 AM
On 10/24/13, 2:20 AM, Girish Sharma wrote:
> I think using the word "Dev" will be really good especially when Chrome
> also uses the word Dev for their build of Chrome which is equivalent to
> Aurora. This means that users who use Chrome will already be familiar with
> the term "Dev" builds and its purpose.
>
> Also, here at DevTools, we also focus on a train-ly basis post highlighting
> new features in Aurora. This will be in line with the new name (if chosen)
> "Dev".

Yes, "Dev" sounds like a reasonable name and follows Chrome's precedent.
We can label the channel "Dev", but call it something like "Developer
Preview" to market the channel to web developers (and away from
non-power users that should use Beta).


cpeterson


Re: Coupled Train Model Steve Wendt 10/24/13 11:08 AM
On 10/24/2013 4:47 AM, Philip Chee wrote:

> I think Thunderbird has a "Early Bird" branch. So perhaps by analogy we
> could use "Early Fox". What are baby foxes called anyway?

http://forum.wordreference.com/showthread.php?t=1177940


Re: Coupled Train Model Mike Connor 10/24/13 11:46 AM
In that case, we can bring this little guy back:

Developer Kit:
https://mdn.mozillademos.org/files/561/Moz_ffx_openStandards_1680x1050.jpg

Aligning what is now Aurora with MDN and web developers feels like it
would be a great way to cross-promote these builds to developers.

-- Mike
Re: Coupled Train Model Robert Kaiser 10/24/13 3:10 PM
Axel Hecht schrieb:
> I don't see room for l10n here?

Localizers would at least have a target that is between what Nightly and
Beta are, with smaller-sized chunks of strings coming in but not on a
daily pace (though of course without a full freeze, which would
presumably only be on the last of those merges).

KaiRo
Re: Coupled Train Model Robert Kaiser 10/24/13 3:11 PM
Philip Chee schrieb:
> On 24/10/2013 06:07, Matt Basta wrote:
>> The name Aurora is the single biggest factor that contributes to it
>> not having many users. I've never met a single non-Mozillian that had
>> any idea what Aurora was or what it was supposed to represent. It may
>> be a cute name and have cute branding but if users don't know what
>> the hell it is, then we've failed. Let's call it something more
>> generic, like Alpha or Early or something like that.
>
> I think Thunderbird has a "Early Bird" branch. So perhaps by analogy we
> could use "Early Fox". What are baby foxes called anyway?

AFAIK "Kit". So it would be "Firekit", I guess. Which sounds like a
Firefox version based on WebKit. :-/

KaiRo

Re: Coupled Train Model Matt Basta 10/24/13 3:34 PM
> Does alpha still really mean "crashes a lot; unusable for daily work"?

"Aurora" certainly doesn't convey that meaning. Aurora doesn't really convey *any* meaning.

----- Original Message -----
From: "Gervase Markham" <ge...@mozilla.org>
To: dev-pl...@lists.mozilla.org
Sent: Thursday, October 24, 2013 1:59:21 AM
Subject: Re: Coupled Train Model

Re: Coupled Train Model chris hofmann 10/24/13 3:56 PM
On 10/24/13 3:34 PM, Matt Basta wrote:
>> Does alpha still really mean "crashes a lot; unusable for daily work"?
> "Aurora" certainly doesn't convey that meaning. Aurora doesn't really convey *any* meaning.
>
>
right,  sort of.  it doesn't covey any meaning related to software.

I'd argue that "beta" has gained traction and widespread awareness as a
state of software in development among the general population. You can
see that in definitions like http://en.wikipedia.org/wiki/Beta#Computing

I'd also argue that "alpha" hasn't gained the same traction, and its
pretty much an "inside baseball" term used among software developers and
not among any class of software users.   See lack of definition in
http://en.wikipedia.org/wiki/Alpha and fewer general use on the web
versus the term "beta."

Firefox Developer Release
Firefox Developer Release
Firefox Developer Release
Firefox Developer Release
Firefox Developer Release
Firefox Developer Release
Firefox Developer Release
Firefox Developer Release

that's who we want to target.

that's who we want using the pre-release versions of Firefox as soon as
the stabilize off central.

that's who we want testing their websites, and checking there addons
before we go for a larger audience of 10 million users.

-chofmann
Re: Coupled Train Model Clint Talbert 10/25/13 8:49 AM
On 10/22/2013 06:11 PM, Lawrence Mandel wrote:
> Right. My suggestion was based on the idea that the 6 weeks on Aurora are not very useful as a time between Nightly and Beta but the more stable Aurora channel is useful to certain groups like QA and l10n and (ideally) Web devs. As such, let's give these groups a stable channel that runs concurrently to Nightly while cutting 6 weeks from the cycle. Note that I haven't discussed this idea with anyone from QA or l10n so I don't know if this will be useful to them. (I expect that l10n will need some sort of string freeze toward the end of the Nightly cycle to make this useful which may or may not be a show stopper.)
What you lose with this model is one of the original intentions of the
train model which was to prevent crash landings. If at the end of a 6
week cycle, right before we move "aurora" to beta we land some new
feature from nightly to aurora then it has no bake time, no
stabilization and no l10n. This effectively means that you can't just
land anything on nightly that's ready; you have to land what is
"stable". And that breaks the entire idea of the train model in the
first place.

The thing that everyone forgets too is that the beta QA and the aurora
QA happen simultaneously. They don't have the luxury of "one, then the
other". QA is trying to re-marshall their efforts to focus first on
aurora in order to achieve a much more stable beta (and thereby be able
to ramp down some QA on beta in order to let the larger user base we
have there QA for us).  That approach works well with our current model.
I'm not sure that approach would work well with the new model at all.

Clint


Re: Coupled Train Model Alex Keybl 10/25/13 8:52 AM
> What you lose with this model is one of the original intentions of the train model which was to prevent crash landings. If at the end of a 6 week cycle, right before we move "aurora" to beta we land some new feature from nightly to aurora then it has no bake time, no stabilization and no l10n. This effectively means that you can't just land anything on nightly that's ready; you have to land what is "stable". And that breaks the entire idea of the train model in the first place.

This isn't the case. We've always allowed for stabilization for 1-2 weeks before moving to Beta in this proposal. We're still working out the specifics of l10n.

-Alex
Re: Coupled Train Model Eric Rescorla 10/25/13 8:56 AM
On Fri, Oct 25, 2013 at 8:49 AM, Clint Talbert <ctal...@mozilla.com> wrote:

> On 10/22/2013 06:11 PM, Lawrence Mandel wrote:
>
>> Right. My suggestion was based on the idea that the 6 weeks on Aurora are
>> not very useful as a time between Nightly and Beta but the more stable
>> Aurora channel is useful to certain groups like QA and l10n and (ideally)
>> Web devs. As such, let's give these groups a stable channel that runs
>> concurrently to Nightly while cutting 6 weeks from the cycle. Note that I
>> haven't discussed this idea with anyone from QA or l10n so I don't know if
>> this will be useful to them. (I expect that l10n will need some sort of
>> string freeze toward the end of the Nightly cycle to make this useful which
>> may or may not be a show stopper.)
>>
> What you lose with this model is one of the original intentions of the
> train model which was to prevent crash landings. If at the end of a 6 week
> cycle, right before we move "aurora" to beta we land some new feature from
> nightly to aurora then it has no bake time, no stabilization and no l10n.
> This effectively means that you can't just land anything on nightly that's
> ready; you have to land what is "stable". And that breaks the entire idea
> of the train model in the first place


This seems a little strong: Chrome runs a train model without an alpha.
Yes, people do crash-land on the development tree and so some care
needs to be taken on the version cut, but I don't think it breaks the
entire model.

-Ekr


>
>
The thing that everyone forgets too is that the beta QA and the aurora QA
> happen simultaneously. They don't have the luxury of "one, then the other".
> QA is trying to re-marshall their efforts to focus first on aurora in order
> to achieve a much more stable beta (and thereby be able to ramp down some
> QA on beta in order to let the larger user base we have there QA for us).
>  That approach works well with our current model. I'm not sure that
> approach would work well with the new model at all.
>
> Clint
>
>
>
> ______________________________**_________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/**listinfo/dev-planning<https://lists.mozilla.org/listinfo/dev-planning>
>
Re: Coupled Train Model Clint Talbert 10/25/13 12:03 PM
Turns out you're right, Eric. I was misunderstanding what we were
discussing and had conflated Alex's original idea with Lawrence's
modification into one model. After chatting some more with Alex, I see
that this is the current model we are proposing (or something pretty
close to it). [1]

Here, you can see that every two weeks we merge nightly to aurora (those
bars on nightly are 1 week). And then after 9 weeks, we stabilize Aurora
and land aurora to beta. Once that happens we resume the every 2 weeks
cadence of merges from nightly to aurora.  I really like this idea since
it will narrow down the amount of code that has to be reviewed and QA'd
on each push to aurora and should enable us to find and fix problems
closer to when developers are writing the code, which is good for
everyone. Going to chat more with QA about how exactly that might work
and what (if any) effect that has on rapid betas and where automation
can further help this process.

I've uploaded a really awful picture that helped me understand the
current proposal in the hope that it might help anyone else that had
incorrectly conflated the two proposals in their head. Lawrence and
Alex, feel free to correct the drawing.

Cheers,

Clint
[1]
http://people.mozilla.org/~ctalbert/train_proposal/multiple-merge-release-process.png
On 10/25/2013 08:56 AM, Eric Rescorla wrote:
>
>
>
> On Fri, Oct 25, 2013 at 8:49 AM, Clint Talbert <ctal...@mozilla.com
>     _______________________________________________
>     dev-planning mailing list
>     dev-pl...@lists.mozilla.org <mailto:dev-pl...@lists.mozilla.org>
>     https://lists.mozilla.org/listinfo/dev-planning
>
>

More topics »