| Fwd: TB Summary of release discussion | Kent James | 11/09/12 15:13 |
At the recent Thunderbird summit in Warsaw, there were
discussions about the future release cycle of Thunderbird. Wayne
prepared the attached spreadsheet trying to summarize this complex
subject, and there are also some meeting notes available at
https://etherpad.mozilla.org/qIkqbtHTSV I'm going to try to
clarify the discussion some, and reflect some understandings as
well from an IRC discussion that we had today.
As a general summary, there are a number of parallel issues. 1) Rate of release of feature editions (mapped to Wayne's options) Option 0: Currently, we do one release per six-week Gecko cycle/7 releases per ESR cycle. This is not going to happen beyond TB 17. Option A: We only do ESR releases (1 release per ESR cycle) Option B: We do 2 releases per ESR cycle. (This is the level of release that Mozilla commits to providing staff support for). (not discussed by Wayne): We do more than two releases per ESR cycle. Standard8 in earlier memos discussed this as an option that would be possible with "community support". From the development perspective, the main issue is whether we allow intermediate releases at all, so if we do option B) it is only a resource question whether to add more intermediate releases in the future. So I will redefine B) to mean "two or more releases per ESR cycle". Wayne listed an option C) and D) that mentioned some additional issues here, but in IRC we clarified those issues. This obsoletes the concept of a separate C) and D) option, and the issues discussed in C) and D) are discussed in the various issues below. If we choose option A), then most other issues go away. That is, new work is landed on central (following current Gecko), central->aurora->beta->release transitions occur as now every 6 weeks (but the release channel does not get updated from the release repo every six weeks). The rate of new beta on the beta channel is reduced to about once per Gecko cycle except as we approach TB 24, when that rate may increase. The current release channel is identical to the current esr channel, but is maintained only to allow us to reconsider this decision later. Assuming though that we choose B) and allow intermediate releases, a number of additional issues come up: 2) What Gecko repository do we use for intermediate feature releases? Earlier there was some discussion about not using the Gecko ESR repository. I think there is a consensus that using the ESR Gecko repository for all future releases is the only viable approach. 3) Do we create a separate parallel intermediate feature branch only on demand and then backport desired features with an a+ patch approval, or do we create this branch immediately and land most new features simultaneously on both trunk and on the release branch (probably without needing an a+ approval initially)? Let me use an "fr" suffix for the feature release branch. If we do the "create immediately" option, it seems to me the repo and channel flows would go like this: central-fr ==> aurora-fr ==> beta ==> release | | | -> release update channel -> beta update channel central ==> aurora | | | -> aurora update channel (daily) -> daily update channel (daily) After the last intermediate release, we abandon central-fr && aurora-fr, and update aurora==>beta I'm less clear of the "on demand" variation. Perhaps like this: central ==> aurora ==> beta (with l10n complete) => release | | | -> aurora update channel -> daily update channel comm-fr is created on demand from comm-esr. Patches with a-tb-next+ are landed. It might be good to have an additional channel with completed localization, maybe earlybird-l10n, which would come from the beta comm-fr | | | -> release update channel |-> beta update channel DISCUSSION: It seems to me that in either case, there is a serious deterioration in the available testing channels at a time when our paid staff support for support and QA will be dramatically reduced. In both options, there is no prior testing of the beta channel on the immediately upstream repo, so that channel will be less stable than in the past, which will drive testers away. I think that we are going to be hard pressed to keep up with quality in the new model in any case. One path to failure would be if future releases start to have serious bugs and regressions. That would kill us faster than slow release of features. So I think that the quality risk, as well as the extra effort, of the intermediate feature release is not worth the benefit. I would rather encourage early adopters to use a beta channel which has new features instead, and maintain a stable product at release. In either case, SeaMonkey would I assume use the release repo for their releases. The new model gives them the possibility to approve patches for that channel independently of TB, except near ESR releases. :rkent -------- Original Message --------
Attached please find spreadsheet summarizing, I hope, all the options and very compactly the issues, pros and cons. I have not attempted to include all comments for reasons of space, but they are very nicely noted in the etherpad https://etherpad.mozilla.org/qIkqbtHTSV by our wonderful note takers. Thanks to Kent and Mark who went over an early draft. If I have missed any options or any important issues in the summary please let me know and I will amend the document. W. | ||||||
| Re: TB Summary of release discussion | el.cameleon | 12/09/12 00:17 | Thanks for this post Kent. I want to add that the feeling on the French forum is also that just 1 release a year is a good idea. No need for extra releases, especially if there is a beta channel which should be as stable as the current release channel (in my opinion)... - Normal user will go to the ESR channel - Advanced users will go to the beta channel (Thunderbird Feature Edition) We should have less work on support's forum, because no new feature means less new questions. This is also a good point. -- Vincent (caméléon) | ||||||
| Re: Fwd: TB Summary of release discussion | Kent James | 14/09/12 21:12 | I have an alternate proposal for the release planning.
One issue that we are facing now, approaching ESR for TB 17, is that we are discovering issues too late for any l10n work to be done. We are going to face that issue much worse in the ESR TB-24 time frame. So my proposal is that we do "intermediate releases" to the main release channel starting at either TB 22 or TB 23. These would be releases from the main central/aurora/beta/release repositories so would not need additional repos with all of the complications of that. Two big advantages: 1) We get one or two cycles of testing with the release channel before we lock down a ESR release. 2) We get one or two cycles of l10n that we could use to fix any issues that are discovered. :rkent _______________________________________________ tb-planning mailing list tb-pl...@mozilla.org https://mail.mozilla.org/listinfo/tb-planning | ||||||
| Re: Fwd: TB Summary of release discussion | Kent James | 14/09/12 22:14 | On 9/14/2012 9:30 PM,
Unicorn.Consulting wrote:
OK I like that - and it could be done for the 17 series as well. So to interpret what I think you are saying in terms of repos and channels, we proceed with 17.0 as planned - but it is not the ESR release. We freeze mozilla-central in comm-aurora, release, and beta so that eventually release, beta, and aurora are all on gecko17. New work is landed in comm-central (with current gecko), and selected patches are landed in aurora (with a+) as needed in 17.1 (which will also be ESR). After 17.1 releases, we restart the central->aurora->beta migrations as now (but no new releases until 24.0 then 24.1 ESR) So we do the allowed 1 intermediate release, but we do it immediately so that hopefully central and aurora are close enough together that the backport is easy. If that is your proposal, I like it! :rkent | ||||||
| Re: Fwd: TB Summary of release discussion | Unicorn.Consulting | 15/09/12 16:20 |
Your interpretation is it I never really got the channels but it
sounds exactly what I was trying to say. The only one that roles
along undaunted is nightly and the point 1 release never contains
new features, only bug fixes (although they are one and the same
sometimes) I think also that it lives up to the intent better.
ESR is really aimed at those wanting tried and tested products, to
deliver on that expectation we should not be offering new features
to them at the time of release.
-- “Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety.” Benjamin Franklin | ||||||
| Re: Fwd: TB Summary of release discussion | Mark Banner | 18/09/12 01:00 | On 11/09/2012 23:13, Kent James wrote:... > Assuming though that we choose B) and allow intermediate releases, aThe thought here was that we would use something like a *-esr17.1 set of repositories. The procedure would be to land on central and esr17.1 at least 12 weeks before the intermediate release, which would give time for l10n to do their work when the feature got to aurora. The beta channel would be hijacked from the *-beta repositories onto the *-esr17.1 ones straight after 17 is released, which would give the beta coverage necessary for the features landing in the 17.1 repo. Once 17.1 is released, the beta channel would revert to the existing *-beta repositories. The above model could be tweaked, so that the feature/function would only land on the esr17.1 repos after it has migrated to aurora/beta in the main repositories, which I think would alleviate some of these issues. > us faster than slow release of features.So I think that the quality > risk, as well as the extra effort, of the intermediate feature releasePersonally, I think the beta should only be recommended for testing new features, as has been previously mentioned - we need to be careful to set the expectations of quality correctly. Whilst regressions during an intermediate release would concern me, I'd also expect there to be a significantly smaller amount of changes compared to a new gecko-based release, especially as we can do some picking and choosing of what we believe will be stable. Mark. | ||||||
| Re: Fwd: TB Summary of release discussion | Mark Banner | 18/09/12 01:11 | Assuming these are based on the equivalent Gecko versions, then we would also need to consider back-porting of the core security fixes to those releases for each of the .0.1 releases. ESR has been designed with new features at the time of the ESR being considered. There is an overlap of 2 releases on the ESR channel to allow organisations time to switch over and for any critical issues to be resolved. The ESR FAQ has a diagram showing this overlap. Hence, this allows for the .0.1 and .0.2 releases. I think my other message to this thread covers this slightly differently. What I think I could do with though, is a clear statement of the problem you're trying to solve here, as this doesn't just sound like just allowing an intermediate release of features. Earlier in the thread you mentioned there's issues fixing bugs due to L10n. Are these flagged as tracking-thunderbird17? Can you also provide some examples? Thanks Mark. | ||||||
| Re: Fwd: TB Summary of release discussion | Gervase Markham | 18/09/12 02:19 | On 18/09/12 09:00, Mark Banner wrote:At yesterday's MoCo meeting, it was suggested that B2G are going to need to do releases faster than ESR but slower than rapid release. It _could_ be that this will result in longer-lived maintained Gecko branches that we could use. Of course, it could be that those branches carry too many B2G-specific hacks for us to use. Just an idea... Gerv | ||||||
| Re: Fwd: TB Summary of release discussion | Kent James | 18/09/12 09:03 | At the moment I prefer Matt's proposal and not this one. But in defense of my old position, all that I was suggesting is that we do the "intermediate release" one or two cycles prior to the ESR release, and that we use the same procedures that we have in place today to do those. I would do a TB22 and TB 23 or just a TB 23. In neither case do we need backporting, just like we don't need backporting today - we just start the old rapid release train one or two releases before ESR. I think that the concern that Matt and I are showing is that each new *. release seems to come with a few new critical issues that are not resolved until the *.0.1 or *.0.2 release. Currently ESR 17 is the same as TB17 and has the same initial instability. Why is that needed? Why not delay the ESR release relative to the main release until some period of time has elapsed for those instabilities to get worked out? ESR has no rush to it. The problem we are trying to solve is preventing the *.0.0 instabilities from hitting the ESR channel, so that ESR users do not have to go through a period of pain before the release stabilizes. Matt's proposal does this better than my initial proposal. The current example of a bug where potentially better fixes were not considered due to L10n constraints is "[Bug 791311] Don't disable the menubar for existing users", thought that is just a current example. The more general point is that we should not let L10n constraints prevent us from presenting ESR users with the optimal fixes for bugs. That may or may not be an issue in the current ESR release, if not then ESR could happen at 17.0.1 or 17.0.2 Maybe you are doing that anyway. :rkent | ||||||
| Re: Fwd: TB Summary of release discussion | Mark Banner | 19/09/12 01:31 | I can understand the concern, but I don't believe it is really critical to ESR. Despite our mentions about testing the changes, I'm pretty sure some organizations won't start widely testing until the next ESR is actually released. So if there are issues that affect them specifically, they may not get reported until we release the ESR. If we delayed releasing that ESR for a couple of point releases, then they would be in the case where the old ESR version would be unsupported, but they might not be able to upgrade to the new version due to the issues. The overlap is there for exactly this reason - upgrade straight away if you can, but if not, you've got 12 weeks to resolve any issues (or get us to resolve them). Something else to point out here I think, is that when we release ESR 17.0, only syadmins that choose to upgrade their uses to the new ESR version will do so. As there will be two more ESR 10.0 releases (one in sync with ESR 17, the second in sync with 17.0.1), I don't see us prompting 10.0 users to do a major upgrade until 17.0.2 is released and 10.0.x is obsolete. Just as an example, for specifically for an ESR, we might want to have an option whereby sysadmins can choose that Thunderbird does not automatically remove the menubars (most sysamdins have set-ups that allow them to set default prefs etc). We've done this type of thing before for them and it may be appropriate here. Mark. | ||||||
| Re: Fwd: TB Summary of release discussion | Kent James | 19/09/12 07:59 | That's a good point. I had not thought of the issues of the overlap. OK that's news to me, thanks for pointing that out. With this discussion, I think that I am going back to my earlier position, that the best way to use an intermediate release would be to release a Thunderbird 23 or 22, so that we would get some additional user feedback on issues before we get locked in for another year. What do you think, Matt? :rkent | ||||||
| Re: Fwd: TB Summary of release discussion | Unicorn.Consulting | 19/09/12 16:10 |
I have no real objection.
| ||||||
| Re: Fwd: TB Summary of release discussion | Ludovic Hirlimann | 20/09/12 05:45 | So you guys aren't worried that 6 or 12 weeks would be very short to gather feedback bubble it up and have things fixed ? I'd rather have 20 or 21 as a target train. -- @lhirlimann on twitter https://wiki.mozilla.org/Thunderbird:Testing my photos http://www.flickr.com/photos/lhirlimann/collections/ | ||||||
| Re: Fwd: TB Summary of release discussion | Kent James | 20/09/12 08:52 |
Oh yes, I'm worried. I think I've said before that the future of
Thunderbird will be determined by Thunderbird 24, and there are
reasons to worry.
But it is also true that we are constrained by resources, and releases take a lot of resources. Also we need to see the effect of Gecko changes to really get stable, and those arrive late. So given those constraints, I think that we will have to see in the 20 or 21 time frame where we stand before we can decide - and also leave open the possibility of a 24.1 if needed. But the main point in all of this is that if we can only do one or a few intermediate releases, it is probably better to use those releases to improve that stability of the annual release, rather than to push features to users a little earlier. :rkent |