Proposed Release Plan changes

[Mark Banner]

Hi All,

I just updated the Change of Release and Governance model wiki page with a revised section for Releases, this links straight to a new etherpad containing the initial version of the proposed release plan following the switch to the new model:

https://etherpad.mozilla.org/tb-releases

Sorry for no pretty pictures, I'll put one together after we flesh it out a bit more.

Please add small comments there, and if you think your comment needs a larger discussion, use this thread or start a new one.

Mark.

[Joshua Cranmer]

This definitely merits larger discussion, since I'm kind of attacking the core idea here...

Why go through all of the effort to make release Thunderbird track ESR instead of mozilla-release? The only significant benefit I see is "we don't have to release every 6 weeks," while I see a number of major downsides:
1. Any feature work that gets ported to release Thunderbird can't rely on new Gecko features. This is a category that excludes several major important features that need Gecko changes: seamless iframes win us scrolling headers, several MIME-related fixes; more powerful nsITreeView support wins us bug 213945; and many of our compose issues are actually faults of the editor in Gecko.
2. Any significant change to Gecko since the last ESR creates a major pain port in backporting patches. Some significant changes in the past were XPCOM registration, modifications in testing infrastructure, etc. Changes that we have to look forward to in the near future include things like global renamings (nsnull, PRBool, nsresult, PRInt*, etc.), potential IDL changes, and build system reconfigurations. Many of these might be fixed by straightforward mechanical changes, but every change introduces a small possibility of a bug creeping in; considering that the backporting would be done to a branch with no pre-release baking, this requires the backporters to be extremely eagle-eyed. Note that since Thunderbird has a lot of C++ code, we have this kinds of things anytime someone wants to tweak m-c to remove some of the technical debt.
3. Everyone who wants to maintain compatibility with preview and release versions of Thunderbird now has to contend with the possibility of 8 versions of difference in Gecko (17 on m-esr versus 25 on m-aurora). This introduces similar issues as point #2, but it also applies to community extension authors as well too.
4. From a community engagement perspective, the time from when a patch is submitted to when the patch is visible to most users suddenly becomes horrific. Right now, it takes 3-4.5 months from commit to release. The best-case time remains at 3 months (I'm ignoring things that land more or less simultaneously on c-c/c-a/c-b), while the worst-case time goes to a full year (land on the first day of Gecko 18, get released with 24).

But what would be saved by tracking ESR? comm-central et al still have to track mozilla-central (I'd estimate that, at current rates, you'd have 30,000+ commits between ESR releases and at least 50 m-c-induced bustages too), so you're not saving work there. comm-aurora and possibly comm-beta still serve as user-facing baking branches, so it's not clear to me that a rapid-release-based TB is any less stable or secure than an ESR-based release. Releasing with new features would require somebody to inspect the feature, make sure it's compatible with an older version of Gecko, manually cherry-pick it and backport it across potentially a lot of breaking changes; asking a new contributor to do this can be daunting, so I suspect this task would fall either to a senior contributor or an employee partially dedicated to Thunderbird, potentially increasing the workload in an ESR-based release relative to a rapid-release-based one. Note that I am assuming that contributors will indeed be contributing patch fixes on a moderately regular basis.

I personally am very skeptical that this proposal, which introduces a lot of complexity and potential pain points, is worth the potential savings in maintenance effort.

In short, the questions I want answered are the following (I'm assuming that the intended landing of features is on c-c, not c-r):
1. What are the perceived pros and cons by not adopting the rapid release model?
2. Under this model, who decides what gets ported to ESR-based release?
3. Under this model, who actually ports the features to ESR-based release?
4. Is it expected that most changes (commits other than those necessary to fix m-c-induced issues) will get ported to ESR-based release?
5. Under this model, what would happen if a highly-valued feature arose that relied on a change in Gecko since the last ESR?

-- 
Joshua Cranmer
News submodule owner
DXR coauthor

[Ludovic Hirlimann]

On 8/8/12 11:36 PM, Mark Banner wrote:

Hi All,

I just updated the Change of Release and Governance model wiki page with a revised section for Releases, this links straight to a new etherpad containing the initial version of the proposed release plan following the switch to the new model:

https://etherpad.mozilla.org/tb-releases

Sorry for no pretty pictures, I'll put one together after we flesh it out a bit more.

so

• All releases will be based on Gecko ESR releases

• To provide a more stable core for releases, so that we're not affected so much by Gecko changes

• The ESR model will remain the same, separate from the mainstream channel

Hum why that besides the channel change they'd be identical, I don't see the need to keep two channels with the same product. What's the point in keeping ESR and release based on the same ESR code ?

I don't see the point keeping Aurora and Beta. From the way it currently works we don't benefit much from having Aurora and Beta - most Major regression are found at release time not on Aurora nor beta and if they are they are found too close to the release dates to be actionable. Maintaining Aurora is too time consuming in terms of QA, release management and engineering for US to keep. I do see the l10 arguments but I think we can do it in another way (eg anyway most of the l10n strings from gecko we'll get for free from Ff).

I'de like us to consider the following :

• We base release on ESR
• When we have enough "new features"+bug fixes we cut a beta
• Beta get's tested and released over a 3/4 week period
• Next beta comes out a few months later which gives us time to chew on beta data (bugs, feedback etc ...)

What I'm proposing looks a lot like what happened for 3.0 or 3.1 eg reverting back to pre rapid release. I think this works better with how I see Thunderbird evolving and will give us more stable releases.

Ludo

-- 
@lhirlimann on twitter
https://wiki.mozilla.org/Thunderbird:Testing

my photos http://www.flickr.com/photos/lhirlimann/collections/

[Ludovic Hirlimann]

On 8/9/12 1:41 AM, Joshua Cranmer wrote:

[snip]

> In short, the questions I want answered ar
> e the following (I'm assuming that the intended landing of features is
> on c-c, not c-r):
> 1. What are the perceived pros and cons by not adopting the rapid
> release model?

That's what we currently have. We struggle to deliver things in that
model, when reducing staff to do it struggle will be higher - so quality
will decrease. It seems our suers are ok not getting releases and
updates every 6 weeks and don't mind waiting a bit longer for new features.

> 2. Under this model, who decides what gets ported to ESR-based release?
> 3. Under this model, who actually ports the features to ESR-based release?
> 4. Is it expected that most changes (commits other than those
> necessary to fix m-c-induced issues) will get ported to ESR-based release?

That's how I understand it.

> 5. Under this model, what would happen if a highly-valued feature
> arose that relied on a change in Gecko since the last ESR?

I think it would need to wait to land for the Next major release of
Thunderbird (like it did in the past).

Ludo

--
@lhirlimann on twitter
https://wiki.mozilla.org/Thunderbird:Testing

_______________________________________________
tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning

[Magnus Melin]

On 9.8.2012 02:41, Joshua Cranmer wrote:
> On 8/8/2012 5:36 PM, Mark Banner wrote:
>> Hi All,
>>
>> I just updated the Change of Release and Governance model

>> <https://wiki.mozilla.org/Thunderbird/Proposal:_New_Release_and_Governance_Model>

I've been thinking along the same lines as Joshua. Not following the same
release schedule as Firefox will be very painful. It's not that we can
actually deliver so much new features every 6 weeks that we have to release,
but doing something different from what mozilla-central does goes against the
natural flow, and that hurts a lot.

Doing releases is likely more work than one would guess from the outside, but
doing so could still be money well spent. If we absolutely can't do full rapid
releases, I'd still propose we have those releases as some kind of blessed
builds (hey, even nightlies auto-update!), but make more noise about the
ESR-based releases. From a community engagement perspective it's indeed very
bad if the time until your contribution is live grows further.

-Magnus

[Ben Bucksch]

Joshua,

I agree with all of that from a developer perspective.

From a user perspective, however, people don't want so many releases.
Most people are happy to keep their applications unchanged for years.
People don't /want/ to upgrade, unless the new release has something
that they were waiting for /specifically/ (not generally "better").

Ben

[Ben Bucksch]

On 09.08.2012 08:25, Magnus Melin wrote:
> If we absolutely can't do full rapid releases, I'd still propose we
> have those releases as some kind of blessed builds (hey, even
> nightlies auto-update!), but make more noise about the ESR-based
> releases. From a community engagement perspective it's indeed very bad
> if the time until your contribution is live grows further.

That sounds like a good idea.

I think most users what ESR - in fact they want it even slower than
current ESR.
Some other geek users want the latest and greatest. They currently get
that with the rapid releases, so they use the releases - which is why we
don't find regressions until we have released, as Ludo said.
Essentially, our releases are the betas, which isn't good. So, if we
switch to ESR-based releases as Mark proposed, then make "betas" based
on the rapid releases, and have the nightlies based on m-c for
developers and the really brave users/testers, then we should cover most
bases and get good testing coverage. (This is assuming that making a
beta release isn't much work, because there's very little QA involved.)

[Mark Banner]

On 09/08/2012 00:41, Joshua Cranmer wrote:

This definitely merits larger discussion, since I'm kind of attacking the core idea here...

Thanks for the comments, it obviously needs a bit more explanation as to why we're doing it this way, so here goes. I'm answering via your simplified questions, but I'll answer point 3 now as it isn't covered by those questions.

3. Everyone who wants to maintain compatibility with preview and release versions of Thunderbird now has to contend with the possibility of 8 versions of difference in Gecko (17 on m-esr versus 25 on m-aurora). This introduces similar issues as point #2, but it also applies to community extension authors as well too.

This is already the case with the ESR based on Gecko 10 - authors of extensions already have to keep cross-compatibility if they want to support all/both. I think there will be more stability coming in interfaces, so I don't see this as a huge pain.

In short, the questions I want answered are the following (I'm assuming that the intended landing of features is on c-c, not c-r):

All features, bugs, patches will land on comm-central to begin with (just as they do today), so that they are incorporated in the release that next comes off comm-central. Backporting is optional.

1. What are the perceived pros and cons by not adopting the rapid release model?

• Stability, we don't have significant gecko changing under us every six weeks, nor will we have large Thunderbird changes
  
• We don't have to do so much regression testing continuously
• There's a greater range of time where our changes can get testing
  
• Less risk of regressions that are found after each release, so less likely to need the x.0.1 releases that we've been seeing
• Regressions that have been caused by gecko, don't necessarily have to be fixed straight away before the next cycle, but can be left to run a bit longer
  
• It gives a bit more stability to extension authors and users, as mentioned in the original announcements
• Releases take up a lot of time to track, prepare and release. It isn't just developer time, there's L10n of the website, support documents and more (which generally are a lot of volunteer time). When you're doing releases without significant new features, that doesn't give so much benefit. Security/Stability releases are a lot simpler and can be pushed out in a much simpler manner.
• Time-to-ship features will obviously be increased, depending on when the features land.
• The maximum time would be a year (or something like 42 weeks), but it is still scheduled with a definitive release date.

There's probably a few more, but I think they are the main ones.

2. Under this model, who decides what gets ported to ESR-based release?

I expect this would be a mixture of the community (who want to back port), and the release drivers. The release drivers (as currently) would have the final say as they are responsible for the stability etc.

3. Under this model, who actually ports the features to ESR-based release?

The community - typically we'd expect the person who wrote the feature to do it, but obviously someone else could back-port it as well.

4. Is it expected that most changes (commits other than those necessary to fix m-c-induced issues) will get ported to ESR-based release?

Personally, I would say no. I think intermediate releases based on ESR should only happen when there's a significant set of features for a release. I expect we would however be looking at a slightly less strict security/stability policy for the ESR branch, which would allow for more fixes to be back ported.

5. Under this model, what would happen if a highly-valued feature arose that relied on a change in Gecko since the last ESR?

If there was no-way to work around the needed Gecko change, then the feature would have to wait until the next ESR line, which would be less than a year.

Mark

[JoeS]

Well this has gotten me totally confused.
from the original wiki post:
> There are currently two editions of Thunderbird: 'Thunderbird' and
'Thunderbird ESR'.
> Both will be maintained and based on the same Gecko engine release.

Yet in the etherpad comments:

> Thunderbird Daily builds will be kept up to date and current nightly gecko
>
> Keeping these running gives us:
>
> continous integration with gecko updates avoiding a massive jump
>
> a safe place to land patches/improvements etc for the next major release

If the first quote is correct, this means that the only way mainstream
users will see any community driven, or Gecko core improvements is with
a once a year release with the esr Gecko base.
This separates the everyday user from the dev/testing community by a
full year. What use is community input via bugzilla.
I guess you might say that user impact via bug reports has in fact been
minimal, how about now when we have to say "Someone might take a look at
that, you might see something in next year's release"
Assuming that TB is all that it needs to be for every user could truly
be the death knell for the project.


JoeS

On 8/8/2012 17:36, Mark Banner wrote:
> Hi All,
>
> I just updated the Change of Release and Governance model

> <https://wiki.mozilla.org/Thunderbird/Proposal:_New_Release_and_Governance_Model>

> wiki page with a revised section for Releases, this links straight to a
> new etherpad containing the initial version of the proposed release plan
> following the switch to the new model:
>
> https://etherpad.mozilla.org/tb-releases
>
> Sorry for no pretty pictures, I'll put one together after we flesh it
> out a bit more.
>
> Please add small comments there, and if you think your comment needs a
> larger discussion, use this thread or start a new one.
>
> Mark.
>
>

[Mark Banner]

On 09/08/2012 11:43, Ben Bucksch wrote:
> On 09.08.2012 08:25, Magnus Melin wrote:
>> If we absolutely can't do full rapid releases, I'd still propose we
>> have those releases as some kind of blessed builds (hey, even
>> nightlies auto-update!), but make more noise about the ESR-based
>> releases. From a community engagement perspective it's indeed very
>> bad if the time until your contribution is live grows further.

The problem with "blessed builds" is that you basically need to call
them betas, which is roughly what you get in the proposed plan anyway.

> I think most users what ESR - in fact they want it even slower than
> current ESR.
> Some other geek users want the latest and greatest. They currently get
> that with the rapid releases, so they use the releases - which is why
> we don't find regressions until we have released, as Ludo said.
> Essentially, our releases are the betas, which isn't good. So, if we
> switch to ESR-based releases as Mark proposed, then make "betas" based
> on the rapid releases, and have the nightlies based on m-c for
> developers and the really brave users/testers, then we should cover
> most bases and get good testing coverage. (This is assuming that
> making a beta release isn't much work, because there's very little QA
> involved.)

Betas aren't too much work, we'd probably do one a cycle for most cycles
(assuming no significant issues found for the beta users), except during
the cycles leading up to release when we'd ramp it up again. The only
time we'd do something different would be for intermediate releases,
which would have to have betas based on the Gecko 17 route, but would
include a lot of the fixes from the trunk.

Mark.

[Mark Banner]

On 09/08/2012 04:11, JoeS wrote:
> Well this has gotten me totally confused.
> from the original wiki post:
> > There are currently two editions of Thunderbird: 'Thunderbird' and
> 'Thunderbird ESR'.
>> Both will be maintained and based on the same Gecko engine release.
>
> Yet in the etherpad comments:
>
>> Thunderbird Daily builds will be kept up to date and current
>> nightly gecko
>>
>> Keeping these running gives us:
>>
>> continous integration with gecko updates avoiding a massive jump
>>
>> a safe place to land patches/improvements etc for the next major
>> release
>
> If the first quote is correct, this means that the only way mainstream
> users will see any community driven, or Gecko core improvements is
> with a once a year release with the esr Gecko base.

Both quotes are correct. The plan is that the mainstream and ESR will be
based on the same gecko release - but remember that's Gecko in the first
statement, and we can take (via intermediate releases if there is deemed
to be enough work to warrant them) Thunderbird specific changes later
on. However, unlike previous methods of releasing, we still have a fixed
date to say when a feature will be released.

Obviously, Gecko improvements would be restricted to once a year.

Keeping Daily builds running is just the obvious thing to do - that's
where the next major release comes from, and by keeping them running it
avoids a massive amount of work in uplifting between gecko versions.
Typically, when Gecko breaks us currently (assuming it is something our
builders/test boxes pick up), we get a fairly narrow regression range
and that helps us pick up what changes we need really quickly. If we
don't keep those running, it'd be a lot harder to find all the issues,
especially because we'd likely have multiple failures.

> This separates the everyday user from the dev/testing community by a
> full year. What use is community input via bugzilla.

Yes, this would increase that separation a bit. We had it before though,
and I don't think it will cause us major pains. There may be some more
duplicate reports, but it would also be good to consider what we can do
to improve the handling of that gap. Maybe you could start a separate
thread on that?

Mark.

[Wayne Mery]

actually some (12?) weeks longer than a year, i.e. from feature freeze
to TB24 ship, assuming feature cutoff is when TB24 moves to Aurora.

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

[Wayne Mery]

Quoting Wayne Mery <vsee...@lehigh.edu>:

>
> Quoting Mark Banner <mba...@mozilla.com>:
>
>> On 09/08/2012 04:11, JoeS wrote:
>>> This separates the everyday user from the dev/testing community by
>>> a full year. What use is community input via bugzilla.
>> Yes, this would increase that separation a bit. We had it before
>> though, and I don't think it will cause us major pains. There may
>> be some more duplicate reports, but it would also be good to
>> consider what we can do to improve the handling of that gap. Maybe
>> you could start a separate thread on that?
>>
>> Mark.
>
> actually some (12?) weeks longer than a year, i.e. from feature
> freeze to TB24 ship, assuming feature cutoff is when TB24 moves to
> Aurora.

bah, I said that wrong. *Maximum* (not average of course) deliver lag
for feature TB24 is longer a year, if feature cutoff for the TB17 was
when TB17 moved to Aurora. Put another way, TB24 features start
landing when TB17 moved to Aurora, not when TB17 ships.

[Magnus Melin]

For us not on the release floor - what is it that makes it so much less
work to do a beta vs a release?

-Magnus

[Magnus Melin]

On 09.08.2012 15:01, Mark Banner wrote:

• Stability, we don't have significant gecko changing under us every six weeks, nor will we have large Thunderbird changes
  
• We don't have to do so much regression testing continuously
• There's a greater range of time where our changes can get testing
  
• Less risk of regressions that are found after each release, so less likely to need the x.0.1 releases that we've been seeing
• Regressions that have been caused by gecko, don't necessarily have to be fixed straight away before the next cycle, but can be left to run a bit longer
  

Not having a firm incentive to fix regressions within a 6 week cycle can hardly be seen as a good thing, and will just lead to regressions piling up.

 -Magnus

[Mark Banner]

On 09/08/2012 06:18, Ludovic Hirlimann wrote:

so

• All releases will be based on Gecko ESR releases

• To provide a more stable core for releases, so that we're not affected so much by Gecko changes

• The ESR model will remain the same, separate from the mainstream channel

Hum why that besides the channel change they'd be identical, I don't see the need to keep two channels with the same product. What's the point in keeping ESR and release based on the same ESR code ?

Keeping them separate gives the opportunity for intermediate releases if we have a significant amount of features ready. If we didn't have separate channels, then we wouldn't be able to do those intermediate releases because we'd be breaking the ESR promise. As we don't yet know the amount of contributions that we'll get going forward I feel it would be better to keep the opportunity available rather than take it away. We can always review it in a year or two.

I don't see the point keeping Aurora and Beta. From the way it currently works we don't benefit much from having Aurora and Beta - most Major regression are found at release time not on Aurora nor beta and if they are they are found too close to the release dates to be actionable. Maintaining Aurora is too time consuming in terms of QA, release management and engineering for US to keep. I do see the l10 arguments but I think we can do it in another way (eg anyway most of the l10n strings from gecko we'll get for free from Ff).

Having Aurora will give us an additional slightly-stable channel, but also my main argument would be for L10n - it is where they work from now, and I don't think that it is something worth changing. It also gives them stability, clear checkin needs, and continuous integration rather than big spurts of Thunderbird strings when we decide it is necessary for a release, that something that is much easier to work with, and it has certainly seemed a lot easier to get localisations complete under the rapid release scheme, especially with the improved tools to have now that work with the rapid release model.

From a release management and engineering perspective, I've never found EarlyBird/aurora really time consuming. The builds are there and they run when necessary, whilst it consumes some resources, it is no where as near significant as trunk.

I'de like us to consider the following :

• We base release on ESR
• When we have enough "new features"+bug fixes we cut a beta
• Beta get's tested and released over a 3/4 week period
• Next beta comes out a few months later which gives us time to chew on beta data (bugs, feedback etc ...)

The problem I see here, is that this gives us a semi-random time as to when releases occur, which makes planning for everyone a lot harder. Additionally, if you're looking at a month-cycle for a beta, then I think you'll soon start hitting the place where to get a few betas, you'll actually then be hitting the preparation cycle for the next ESR based release.

What I'm proposing looks a lot like what happened for 3.0 or 3.1 eg reverting back to pre rapid release. I think this works better with how I see Thunderbird evolving and will give us more stable releases.

The problem there is that there is a lot more co-ordination to do with 50+ l10n teams for getting the Thunderbird strings localised and in the right place at the right time. Keeping Earlybird/aurora gives us that with relatively low cost.

Mark.

[Mark Banner]

The main thing to remember is that for a release you need to do a minimum of 2 betas (one at the start of the Gecko cycle to pick up gecko, one at the end to pick up any fixes through the cycle), probably 4 or 5 as you'll want to incrementally include the latest release.

So you've already got several times the effort of pushing out a single beta.

On top of that, for final releases you need to make sure that:

• You've addressed all the bugs you're tracking for the release (for beta we don't necessarily fix all the bugs being tracked as they can be fixed during the cycle)
• Prompt L10n to finish their localisations
  
• Web pages (release notes, system requirements etc) are written and agreed with any blog/press documents prepared
  
• If you have a what's new page, then that needs to be completed and pushed out to localisers (which is basically land n copies of it, file 50+ bugs, and then help out localisers with landing etc)

Mark.

[Joshua Cranmer]

On 8/10/2012 8:57 AM, Mark Banner wrote:

On 09/08/2012 06:18, Ludovic Hirlimann wrote:

so

• All releases will be based on Gecko ESR releases

• To provide a more stable core for releases, so that we're not affected so much by Gecko changes

• The ESR model will remain the same, separate from the mainstream channel

Hum why that besides the channel change they'd be identical, I don't see the need to keep two channels with the same product. What's the point in keeping ESR and release based on the same ESR code ?

Keeping them separate gives the opportunity for intermediate releases if we have a significant amount of features ready. If we didn't have separate channels, then we wouldn't be able to do those intermediate releases because we'd be breaking the ESR promise. As we don't yet know the amount of contributions that we'll get going forward I feel it would be better to keep the opportunity available rather than take it away. We can always review it in a year or two.

Would it be possible to guesstimate what would happen in an "average" release cycle by looking at the changes from TB 10-now?

[Justin Wood (Callek)]

Mark Banner wrote:
>> In short, the questions I want answered are the following (I'm
>> assuming that the intended landing of features is on c-c, not c-r):
> All features, bugs, patches will land on comm-central to begin with
> (just as they do today), so that they are incorporated in the release
> that next comes off comm-central. Backporting is optional.

Shall we be making SeaMonkey-Council drivers of mailnews/ approvals, at
the *least* for non-esr approvals?

Given that SeaMonkey does intend to continue releasing the "normal"
model, and Thunderbird does not. To me that sounds fair as to
risk-vs-reward expectations already inherent in SeaMonkey relman process.

--
~Justin Wood (Callek)

[Justin Wood (Callek)]

Mark Banner wrote:
>> 1. What are the perceived pros and cons by not adopting the rapid
>> release model?
>

> * Stability, we don't have significant gecko changing under us every

> six weeks, nor will we have large Thunderbird changes

CON: But you'll have larger range of changes from ESR to ESR and harder
to pinpoint regression ranges.

> o We don't have to do so much regression testing continuously

Incorrect, SeaMonkey will need the same work if someone breaks things.
(so 'we'==='community' in this case)

> o There's a greater range of time where our changes can get testing

Incorrect if you account that SeaMonkey uses the same code.

> o Less risk of regressions that are found after each release, so

> less likely to need the x.0.1 releases that we've been seeing

Increasing the timeframe, and decreasing users on a release, doesn't
change the RISK of regressions, it does however INCREASE the risk that
it won't be caught before release. And increases the pain of
finding/fixing the regression.

> o Regressions that have been caused by gecko, don't necessarily

> have to be fixed straight away before the next cycle, but can be
> left to run a bit longer

Again, Incorrect if you account for the fact that SeaMonkey is still
doing releases.

> o It gives a bit more stability to extension authors and users, as

> mentioned in the original announcements

ALL the above is true for any code specifically mail/ (or chat/) but not
the shared mailnews/

[Justin Wood (Callek)]

Mark Banner wrote:
> On 09/08/2012 19:23, Magnus Melin wrote:
>> On 09.08.2012 15:52, Mark Banner wrote:
>>> Betas aren't too much work, we'd probably do one a cycle for most
>>> cycles (assuming no significant issues found for the beta users),
>>> except during the cycles leading up to release when we'd ramp it up
>>> again. The only time we'd do something different would be for
>>> intermediate releases, which would have to have betas based on the
>>> Gecko 17 route, but would include a lot of the fixes from the trunk.
>>>
>>
>> For us not on the release floor - what is it that makes it so much
>> less work to do a beta vs a release?
> The main thing to remember is that for a release you need to do a
> minimum of 2 betas (one at the start of the Gecko cycle to pick up
> gecko, one at the end to pick up any fixes through the cycle), probably
> 4 or 5 as you'll want to incrementally include the latest release.
>
> So you've already got several times the effort of pushing out a single beta.

I do agree with your logic in this e-mail. I would however advocate for
2 betas every cycle rather than 1.

"beta 1" (matching Firefox beta 1)
and "beta 2" (matching Firefox Beta 5/6)

This will allow for better regression hunting finding issues for code
that gets backported to beta. (whether it be gecko changes that break
us, or mail changes the community drives)

If you have no intent of doing a full QA run on these betas, it is not
much additional work at all.

--
~Justin Wood (Callek)

[Axel]

+1

So far this has been fairly confusing to me; for the "normal" release channel (if you didn't suggest scrapping it altogether) a house-number like "updates not less than 8 weeks but never more than 4 months" would be helpful.

thanks
  Axel

  Axel

[Mark Banner]

On 10/08/2012 15:14, Joshua Cranmer wrote:

Would it be possible to guesstimate what would happen in an "average" release cycle by looking at the changes from TB 10-now?

We could, but I don't think that would be a fair comparison. Whilst we could take out the work of those who have been paid to work on TB, and just use the rest as a gauge, that doesn't take account of the bias of efforts in recent months where we've picked up new contributors and the possibility that with the new governance in place that we end up with even more contributions than we have now.

Mark.

[Ludovic Hirlimann]

Having numbers now - would help in say 6 month , one year to make change in how the model works.

[Mark Banner]

On 10/08/2012 18:44, Justin Wood (Callek) wrote:
> Mark Banner wrote:
>>> 1. What are the perceived pros and cons by not adopting the rapid
>>> release model?
>>
>> * Stability, we don't have significant gecko changing under us every
>> six weeks, nor will we have large Thunderbird changes
>
> CON: But you'll have larger range of changes from ESR to ESR and
> harder to pinpoint regression ranges.

That's why we want to keep Daily, Earlybird and Beta running as much as
possible with latest Gecko - which will help negate the "large" range of
changes. We know that won't cover everything, but it will go a long way
to helping.

>> o Less risk of regressions that are found after each release, so
>> less likely to need the x.0.1 releases that we've been seeing
>
> Increasing the timeframe, and decreasing users on a release, doesn't
> change the RISK of regressions, it does however INCREASE the risk that
> it won't be caught before release. And increases the pain of
> finding/fixing the regression.

Accepted, but the point here is that we won't be likely to need to be
doing an additional x.0.1 release every six weeks due to a completely
new release on a new gecko, worst-case it'd be once a year.

>> o Regressions that have been caused by gecko, don't necessarily
>> have to be fixed straight away before the next cycle, but can be
>> left to run a bit longer
>
> Again, Incorrect if you account for the fact that SeaMonkey is still
> doing releases.

We have committed to producing two Thunderbird releases a year with the
support of paid staff. We obviously want to do some betas in support of
testing, but if the regression is not major but something we'd fix for
final, we may choose to leave it for a bit longer. Of course, regression
fixing isn't exclusive to paid staff, and can obviously be done by the
community as well.

If the community wanted to do extra Thunderbird releases, or SeaMonkey
wants to do extra releases then the people in the community responsible
for those pieces will need to pick up the work for those releases,
including fixing any regressions.

Mark.

[John Crisp]

On 09/08/12 12:38, Ben Bucksch wrote:
>
> From a user perspective, however, people don't want so many releases.
> Most people are happy to keep their applications unchanged for years.
> People don't /want/ to upgrade, unless the new release has something
> that they were waiting for /specifically/ (not generally "better").

Couldn't agree more.